Most of the time, adding API Gateway does not improve the performance but does add an extra overhead, use API Gateway if any of the additional features it provides are useful to you. If you control the client then can get credentials on the client with permissions to invoke your Lambda functions directly, and you don't want any of the additional API Gateway features then go with the Lambda direct invoke.
Either way, direct lambda calls are faster bcz there's one network hop-less. Maybe you can directly compare lambdas/APIG with regard to the latency of your IAM verification, but we've benchmarked lambda-lambda calls and lambda-APIG-lambda calls, where the lambdas do no work, i.e. simply return the event object.
The average time usually is taken, I am mentioning below:
So the tax for the extra hop is 20 ms. We use lambda-lambda calls whenever feasible, especially with lambdas we don't want to expose to the world.
Let me know if you and your team's able to complete this task, will be happy to help more :)