Story-wise, there are plenty of differences in BioShock Infinite. But the most important one—and easily the best addition in Infinite—is Elizabeth.
She's a companion and a partner, but unlike most tag-along characters in games, Elizabeth is never a burden. She stays out of your way during fights, for one thing. And because Irrational wanted her to feel like a true partner, she'll even toss you ammo, health and other loot during battle.
Beyond that, she's a truly likable character. She's funny, witty and perceptive, but extremely vulnerable at the same time (she did spend most of her life locked in a tower, after all). You'll actually want to spend time with her—and that's the best compliment we can give.