I'm not sure what you mean by huge? The thing's not even half the size of a PS4 and fits comfortably on my TV stand.
Nevertheless I agree that if you're planning on playing primarily in handheld mode, the Switch Lite is probably your better option. It's more affordable, and from what I know, the battery life is better. If I'm wrong on that, someone please feel free to call me out. There are advantages to having the docked version as well, however. It's a much more feature complete experience, for one, and I can't imagine what playing mulitplayer games would be like on the Lite. Again, though, that all hinges on whether or not you even plan on using those features.