It's possible that ACF is fine and there's something else going on, but the more layers you have, the less control and the more potential issues. When you have people using visual scripting tools over third-party frameworks over Unity...yeah.

I was playing Dungeons 2 a while ago and that has the feel that people complain about with Unity games...heavy and clunky. But lately I've been playing The Pedestrian, which has some reasonably complex scenes that the camera moves around in, with dynamic shadows and so on, and it's never dropped a frame or spun up my GPU fans at all.

I totally understand that in this case, it is indeed the only valid and constructive approach ;-)
I think it's probably safe to say here that you give a good example on implementation: one going sideways and one keeping he road.