So, I live where there are no barns...someone explain to me how pole barns work?

This is what I've gathered...Is it accurate?
You have big beams/poles that are the exterior wall/roof support for the structure. Do you not have to do the 2x4 framing between the poles?
They're sheathed directly. Can you use this same building style for homes (Thinking garage w/ apartment on 2nd)? Would you have to add extra wall framing? Would it save any expense/time? Is this not the same as timberframe or post & beam construction? Am I missing something?