how do i update DOTA2?

Does it show in your library and maybe say "Update Required" next to it? If so, right click Dota 2, pause, and then resume and see if it starts.

3GB's now

Well, the only other thing I can think of is to delete the clientregistry.blob file in your steam folder. Doing this will make steam update its platform, so what I think is happening, is that theres something wrong with your steams platform.