Two irreversible sinks for the desired intermediate
Once the aryllithium has been generated by Li–Br exchange, it is the nucleophile that attacks the metal carbonyl. Adding more nBuLi puts a second, competing nucleophile into the same solution, and that nucleophile attacks the same electrophiles.
Addition to the metal carbonyl. A coordinated CO ligand is electrophilic at carbon. nBuLi adds there to give an acyl–lithium, which on workup gives a butyl ketone or, after further reaction, a butylated carbonyl-derived product. Every equivalent of nBuLi that adds this way is an equivalent that never becomes the desired acylate, and it also consumes metal carbonyl that the aryllithium needed.
Alkylation of the oxy-carbene carbon. The carbene carbon is electrophilic because the metal fragment withdraws electron density and the alkoxy oxygen donates through its lone pair. nBuLi is a competent nucleophile toward that carbon, so it delivers a butyl group and forms the same kind of butyl–carbene adduct that residual nBuBr forms. The side product you were trying to suppress is generated by the reagent you added to suppress it.
Deprotonation and solvent attack
nBuLi is also a strong base. It can deprotonate acidic sites on the substrate or on partially formed intermediates, and it slowly attacks ethereal solvent such as THF by removing a proton adjacent to oxygen, which ring-opens the solvent and consumes the reagent. These pathways are slow at low temperature but they never stop.
What the concentration profile looks like
- Aryllithium: rises during the exchange, then falls as nBuLi and residual nBuBr compete for it and for the carbonyl.
- Oxy-carbene: never accumulates to the level it would reach without the extra nBuLi, because it is alkylated as it forms.
- Desired product: tracks the carbene, so it drops.
- Butylated side products: rise steadily, because their formation is irreversible.
The net effect is that the added reagent is consumed by the very intermediates you need.