To me, paragraphing is more of an art than a science, and there are a number of reasons to choose to break a paragraph--or not to break it.
And like so many things in writing, it's all linked up with other bits of craft.
For me, anyway, the most important aspects of paragraphing are preserving rhythm and clarity and creating emphasis. There's a bunch of other stuff involved in this, of course--clear writing, line of direction, making the sentences and clauses flow into one another in a manner that is not confusing to the reader. (By line of direction, I mean leading the reader through the flow of action in a manner that he can follow easily. A logical progression, as it were.)
But for a minute, permit me the indulgence of pretending I can talk about one aspect of craft without talking about all of them.
Paragraph breaks do a bunch of stuff. Sometimes all at once, and sometimes in smaller bits. They can be there for rhythm, to separate ideas, to create emphasis. They can help you avoid stepping on good lines. They can provide pauses for thought or breath, or, if you run a bunch of them together, they can create a kind of breakneck pace and encourage the reader to skim.
So, paragraphing choices revolve around managing that. They also require managing the rhythms of the prose, and realizing that the first and last sentence of a paragraph will get more reader attention than the ones in the middle. They require figuring out where the flow and the line of direction break, and using the paragraph breaks to hook the reader forward across those breaks--like shot cuts in a movie within a scene.
Unfortunately, there's not just one rule of thumb. It's more an art than a science. So, you just try a bunch of stuff until you find things that work.
Practice again seems to be the way to learn. Listen for it, play with it, see what works and what doesn't. Like punctuation, paragraphing is there to assist the reader in understanding the narrative. They provide a written convention for implying verbal inflection, hesitations and pauses. They are there to assist in understanding.
And that's all writing is. It's communication. And the structures we associate with language have evolved to assist that communication.