While using some of the AWS modules for Ansible, you may have been bitten by 1 of these 2 errors:
If we were to walk through a set of tasks in a role to deploy a VPC, we would see a common set of steps.
Each of the modules above can make 1+ calls to AWS and depending on the amount of calls that is being made. AWS will start to throttle the requests it receives and your playbook will fail.
Since a good portion of the modules for aws do not implement a backoff decorator, we are forced to do the following.
Here are a few of the AWS modules that have implemented a basic retry functionality.
AWSRetry.backoff decorator will retry on the following errors.
If an exception that is not in that list is not matched, it will then just raise the exception as it normally would.
The AWSRetry.backoff decorator will retry to call the failing function using an exponential backoff algorithm. Each time the decorated function/method throws an exception, the decorator will wait for x amount of time and retry calling the function until the maximum number of tries is reached. If the decorated function fails on the last try, the exception will occur unhandled.
AWSRetry is derived from the CloudRetry class. This class is meant to be used as a base class to other cloud providers, that want to build similiar functionality into Ansible Cloud modules.
You can the the AWSRetry decorator on GitHub in the Ansible Core repo (Still in a PR) or in my personal repo.
If you do not want to wait, you can just copy module_utils/cloud.py and module_utils/ec2.py into your ansible installation directory.
If the community is going to seriously consider using Ansible for all of it’s cloud provisioning needs, then a backoff decorator needs to be implemented.