The main cause is moulting. Birds’ feathers are amazingly tough, but they do wear out. During nesting there’s the physical damage caused by repeated squeezing in and out of small spaces; crowded nests are also good breeding grounds for parasites that may eat feathers or cause them to fall out. You may spot some very scruffy, even balding, individuals at this time. Long tails may be completely folded over.
So, as soon as their chicks can manage without them, many adults shed their old feathers and grow a new set. Summer is a good opportunity, with warm weather and reasonably abundant food to provide the energy needed. Unlike ducks, which do the whole moult at once and lose the ability to fly, garden birds replace feathers gradually. But, even so, moulting makes them less manoeuvrable and less able to avoid predators. So most stay, as far as possible, in hedges or undergrowth, out of our sight as well. Singing would draw attention, and is less important once they aren’t defending a nesting territory, so that tends to stop too.
Some species have a surprisingly quick family breakdown, with fledglings becoming competitors for territory and food soon after they become independent. These may be actively driven away from the nesting location. Even if things aren’t that drastic, late summer countryside may well provide better feeding and shelter than gardens, so both adults and youngsters may move away from our homes.