After undergoing a violent baptism at the hands of a zealous asshole, you'll awaken in a fountain surrounded by statues that bear names like Father Washington and Father Franklin. It's at this point that you're first alerted to BioShock Infinite's other main theme: American Exceptionalism. The people of Columbia take this as far as literally worshipping the founding fathers.
Thankfully, Booker is a voice of reason, murmuring to himself, "Just because the city flies don't mean it ain't got its fair share of fools." He's no silent protagonist, after all.
Here you'll finally get to walk through the city and meet its inhabitants. For the most part, they're just going about their business and enjoying the celebration of the anniversary of the founding of Columbia. Some hints of crazy slip through, though, and from the get-go it's clear that something weird is going on under the surface of Columbia.
This is also where you're introduced to several important concepts: voxophones, audio records that help flesh out the story like in past BioShock games, lie scattered about, though not as frequently as in Rapture. Technology is also more advanced in Columbia than it should be; mechanical horses seem to work by magic, drinkable vigors grant you incredible powers, and there's even a half-man, half-machine cyborg called the Handyman. Needless to say, you'll get to fight him later.
After putzing around with several carnival games—which themselves reveal more hints at the story, mentioned the Vox Populi resistance movement and "the anarchist Daisy Fitzroy"—you'll continue on your quest.