Once you accept that a composite is a stack of operations, the practical question is: which operation is on top?
Rule: the outer function is the last operation performed on the whole expression.
A reliable way to find it is to ask what you would do last if you were evaluating the expression by hand. That final action is the outer function, and everything it acts on is the inner function.
Check it on a few expressions:
- \(\sin(3x^2)\): the last thing you do is take a sine, so the outer function is \(\sin(\square)\) and the inner function is \(3x^2\).
- \((x^2+1)^5\): the last thing you do is raise to the fifth power, so the outer function is \((\square)^5\) and the inner function is \(x^2+1\).
- \(e^{2x}\): the last thing you do is exponentiate, so the outer function is \(e^{\square}\) and the inner function is \(2x\).
- \(\ln(5x+1)\): the last thing you do is take a logarithm, so the outer function is \(\ln(\square)\) and the inner function is \(5x+1\).
Notice that in every case the outer function is written with a placeholder, and the inner function is exactly what fills that placeholder. That placeholder view is what keeps the two roles from getting mixed up.
A common failure is scanning left to right and grabbing the first thing you see. Left-to-right reading finds \(\sin\) first in \(\sin(3x^2)\), which happens to be correct here, but it finds \(x^2\) first in \((x^2+1)^5\), which is the wrong layer. The last-operation test does not have that failure mode.