Short answer compilations:
- The technical capability of DX10 compared to DX9 doesn't gain you too much, but from DX9 to DX11 there is enough to justify a shift.
- Game engine developers viewed DX10 as just another part of Windows Vista to be skipped, but now that the API has matured into DX11, people are jumping on board.
- By the time game engines were very well tweaked and optimized for DX9, DX11 was in beta, so why not learn the newest? DX10 imposed some pretty strict new standards for veteran DX9 shops, but everyone realizes if you want to make games supporting the newer GPU features you don't really have too much a choice but learn the new API.