{"id":19587,"date":"2023-10-10T05:40:30","date_gmt":"2023-10-10T05:40:30","guid":{"rendered":"https:\/\/www.insentragroup.com\/au\/insights\/uncategorized\/automating-the-resynchronisation-of-constructed-inventories-in-the-ansible-automation-platform\/"},"modified":"2024-12-13T02:22:29","modified_gmt":"2024-12-13T02:22:29","slug":"automating-the-resynchronisation-of-constructed-inventories-in-the-ansible-automation-platform","status":"publish","type":"post","link":"https:\/\/www.insentragroup.com\/au\/insights\/geek-speak\/modern-workplace\/automating-the-resynchronisation-of-constructed-inventories-in-the-ansible-automation-platform\/","title":{"rendered":"Automating the Resynchronisation of Constructed Inventories in the Ansible Automation Platform"},"content":{"rendered":"\n<p>Ansible Automation Controller constructed inventories are a new feature that allows you to create a dynamic inventory from a list of input inventories. You can use this feature to run jobs against hosts and groups from multiple inventories and apply custom logic and filters to them. For example, you can create a constructed inventory that combines hosts from AWS, Azure, and VMware, and add variables and groups based on their attributes.&nbsp;<\/p>\n\n\n\n<p>To create a constructed inventory, you need to specify the input inventories, the limit pattern, and the source vars. The input inventories are the existing inventories that provide the inventory content for the constructed inventory. The limit pattern is an optional parameter that allows you to filter the hosts using the standard Ansible host pattern syntax. The source vars is a YAML dictionary that defines the logic for adding groups and variables to the constructed inventory. You can use Ansible-style Jinja2 expressions and filters in the source vars.&nbsp;<\/p>\n\n\n\n<p>The constructed inventory is refreshed every time you run a job that uses it. Ansible Automation Controller runs the ansible-inventory command with the constructed plugin and the parameters you provided and generates a temporary inventory file for the job. This ensures that the constructed inventory always reflects the latest state of the input inventories and your custom logic.&nbsp;<\/p>\n\n\n\n<p>There are a few limitations to constructed inventories:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>They are currently only supported in Ansible Automation Platform (AAP) 2.4 and higher&nbsp;<\/li>\n\n\n\n<li>They cannot be used as input inventories for other constructed inventories&nbsp;<\/li>\n\n\n\n<li>If the input inventories are large or complex, the refresh process can take some time&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>The refresh\/resync process for the constructed inventory activates whenever its associated template launches, potentially extending the overall duration of the job more than necessary. While constructed inventories allow manual resync and support caching, manual intervention remains the sole alternative for resynchronisation in case you want to run an ad-hoc job or you want to verify the hosts and groups. As of AAP 2.4.x, there&#8217;s no built-in scheduling feature for constructed inventories within the GUI.&nbsp;<\/p>\n\n\n\n<p>Although scheduled resync is not supported in the current version, the API does allow for source updates to the Constructed Inventories. You can execute the API call using the URL link&nbsp; below:&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>https:&#47;&#47;aap01.example.net\/api\/v2\/inventories\/386\/update_inventory_sources\/<\/code><\/pre>\n\n\n\n<p><strong>Note<\/strong>: 386 in the above URL is the inventory id of one of the constructed inventories in your environment.&nbsp;&nbsp;<\/p>\n\n\n\n<p>To update multiple constructed inventories, one can leverage Ansible Code alongside the ansible.controller collection (available to those with access to console.redhat.com) or the awx.awx ansible collection.&nbsp;<\/p>\n\n\n\n<p>This blog outlines two methods for updating Constructed Inventories. Because the hosts and groups within the constructed inventory rely on the source inventory and its associated query, it&#8217;s essential first to refresh the dynamic inventory serving as the source before synchronising the constructed inventory.&nbsp;<\/p>\n\n\n\n<p>I am going to describe the code necessary to update all the constructed inventories first&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Ansible Code Description: aap-inv-refresh role&nbsp;<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>Directory Structure: \n<br>\naap-inv-refresh\/ \n<br>\n\u2502 \n<br>\n\u251c\u2500\u2500 aap-inv-refresh\/ \n<br>\n\u2502   \u2514\u2500\u2500 tasks\/ \n<br>\n\u2502       \u251c\u2500\u2500 inv_refresh.yml \n<br>\n\u2502       \u2514\u2500\u2500 main.yml \n<br>\n\u2502 \n<br>\n\u251c\u2500\u2500 README.md \n<br>\n\u251c\u2500\u2500 site.yml \n<br>\n\u2514\u2500\u2500 vars\/ \n<br>\n    \u251c\u2500\u2500 aap_vars.yml \n<br>\n    \u2514\u2500\u2500 creds.yml <\/code><\/pre>\n\n\n\n<p><strong>Purpose:<\/strong>&nbsp;<\/p>\n\n\n\n<p>The Ansible role aap-inv-refresh is designed to refresh constructed inventories on Ansible Automation Platform (AAP).&nbsp;<\/p>\n\n\n\n<p><strong>Key Files and Content:<\/strong>&nbsp;<\/p>\n\n\n\n<p><strong>site.yml<\/strong>: Main playbook file.&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>--- \n<br>\n- name: Refresh constructed inventory \n<br>\n  hosts: all \n<br>\n  roles: \n<br>\n    - role: aap-inv-refresh <\/code><\/pre>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>vars\/aap_vars.yml<\/strong>: Contains variables specific to AAP and configured Inventories:&nbsp;<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code># defaults for aap-gcp \n<br>\nee_validate_certs: false \n<br>\naap_controller_host: aap01.example.net \n<br>\naap_organisation: SharedInventory \n<br>\nconstructed_inventories_list: \n<br>\n  - INV-CONSTRUCTED-AZURE-LINUX-ALL \n<br>\n  - INV-CONSTRUCTED-AZURE-OEL-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-RHEL-NONSAP \n<br>\n  - INV-CONSTRUCTED-AZURE-RHEL-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-SLES-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-WINDOWS \n<br>\n  - INV-CONSTRUCTED-AZURE-WINDOWS-SAP \n<br>\n  - INV-CONSTRUCTED-GCP-RHEL \n<br>\n  - INV-CONSTRUCTED-GCP-WINDOWS \n<br>\ngcp_shared_inventory: INV-SHARED-GCP-ALL <\/code><\/pre>\n\n\n\n<ol class=\"wp-block-list\" start=\"2\">\n<li><strong>vars\/creds.yml<\/strong>: <em>The <\/em>creds.yml file can be encrypted with ansible-vault. It contains credentials for authentication with AAP.&nbsp;<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code># AAP credentials \n<br>\naap_controller_username: svc_ansible_token \n<br>\naap_controller_password: svc_ansible_token <\/code><\/pre>\n\n\n\n<ol class=\"wp-block-list\" start=\"3\">\n<li><strong>Tasks<\/strong>:&nbsp;\n<ul class=\"wp-block-list\">\n<li><strong>main.yml<\/strong>: Contains tasks to include default variables, query the inventory by its name, update all inventory sources, and refresh tasks for constructed inventories.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>--- \n<br>\n- name: Include default extension files in vars \n<br>\n  ansible.builtin.include_vars: \n<br>\n    dir: \"{{ playbook_dir }}\/vars\" \n<br>\n    extensions: \n<br>\n      - 'yml' \n<br>\n  tags:  \n<br>\n    - azure_refresh \n<br>\n    - gcp_refresh \n<br>\n <br>\n<br>\n- name: Query the inventory by its name \n<br>\n  set_fact: \n<br>\n    inventory_id: \"{{ (query('ansible.controller.controller_api', 'inventories', host=aap_controller_host, username=aap_controller_username, password=aap_controller_password, query_params={ 'name': gcp_shared_inventory }) | first).id }}\" \n<br>\n    inventory_org: \"{{ (query('ansible.controller.controller_api', 'inventories', host=aap_controller_host, username=aap_controller_username, password=aap_controller_password, query_params={ 'name': gcp_shared_inventory }) | first).summary_fields.organization.name }}\" \n<br>\n    inventory_name: \"{{ gcp_shared_inventory }}\" \n<br>\n  tags:  \n<br>\n    - gcp_refresh \n<br>\n<br> \n<br>\n<br> \n<br>\n- name: Update all inventory sources \n<br>\n  ansible.controller.inventory_source_update: \n<br>\n    name: \"{{ item }}\" \n<br>\n    inventory: \"{{ inventory_name }}\" \n<br>\n    organization: \"{{ inventory_org }}\" \n<br>\n    controller_username: \"{{ aap_controller_username }}\" \n<br>\n    controller_password: \"{{ aap_controller_password }}\" \n<br>\n    validate_certs: \"{{ ee_validate_certs | default(omit) }}\" \n<br>\n    controller_host: \"{{ aap_controller_host | default(omit) }}\" \n<br>\n    wait: true \n<br>\n  loop: \"{{ query('ansible.controller.controller_api', 'inventory_sources', host=aap_controller_host, username=aap_controller_username, password=aap_controller_password, query_params={ 'inventory': inventory_id }, return_ids=True ) }}\" \n<br>\n  tags:  \n<br>\n    - gcp_refresh \n<br>\n  ignore_errors: true \n<br>\n  no_log: true \n<br>\n<br>         \n<br>\n- name: Refresh Task \n<br>\n  include_tasks: inv_refresh.yml \n<br>\n  loop: \"{{ constructed_inventories_list }}\" \n<br>\n  tags:  \n<br>\n    - azure_refresh \n<br>\n    - gcp_refresh <\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>inv_refresh.yml<\/strong>: Contains tasks to query specific inventories by their name, display the ID of the queried inventory, and update all the inventory sources.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>---                                                                                                                                                                                                                                                         \n<br>\n- name: Query the inventory by its name \n<br>\n  set_fact: \n<br>\n    inventory_id: \"{{ (query('ansible.controller.controller_api', 'inventories', host=aap_controller_host, username=aap_controller_username, password=aap_controller_password, query_params={ 'name': item }) | first).id }}\" \n<br>\n    inventory_org: \"{{ (query('ansible.controller.controller_api', 'inventories', host=aap_controller_host, username=aap_controller_username, password=aap_controller_password, query_params={ 'name': item }) | first).summary_fields.organization.name }}} \n<br>\n\" \n<br>\n    inventory_name: \"{{ item }}\" \n<br>\n  tags: \n<br>\n    - azure_refresh \n<br>\n    - gcp_refresh \n<br>\n <br>\n<br>\n- name: Display the ID of the queried inventory \n<br>\n  debug: \n<br>\n    msg: \"{{ inventory_name }} has inventory_id: {{ inventory_id }} and is in organization: {{ inventory_org }}\" \n<br>\n  tags: \n<br>\n    - azure_refresh \n<br>\n    - gcp_refresh \n<br>\n <br>\n<br>\n <br>\n<br>\n- name: Update all inventory sources \n<br>\n  ansible.controller.inventory_source_update: \n<br>\n    name: \"{{ inventory_source_item }}\" \n<br>\n    inventory: \"{{ inventory_name }}\" \n<br>\n    organization: \"{{ inventory_org }}\" \n<br>\n    controller_username: \"{{ aap_controller_username }}\" \n<br>\n    controller_password: \"{{ aap_controller_password }}\" \n<br>\n    validate_certs: \"{{ ee_validate_certs | default(omit) }}\" \n<br>\n    controller_host: \"{{ aap_controller_host | default(omit) }}\" \n<br>\n  loop: \"{{ query('ansible.controller.controller_api', 'inventory_sources', host=aap_controller_host, username=aap_controller_username, password=aap_controller_password, query_params={ 'inventory': inventory_id }, return_ids=True ) }}\" \n<br>\n  loop_control: \n<br>\n    loop_var: inventory_source_item \n<br>\n  tags: \n<br>\n    - azure_refresh \n<br>\n    - gcp_refresh <\/code><\/pre>\n\n\n\n<p>Note: There&#8217;s potential to enhance the code by saving the results from an initial query into a variable and then conducting local searches on that result. I&#8217;ve observed that performance may vary based on the number of inventories present in the environment. In contexts with hundreds of inventories, the resulting JSON file can be substantially large, potentially impacting the solution&#8217;s efficiency. In such cases, I prefer utilising multiple API calls, as each call yields a single-page result.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Resync for Azure&nbsp;<\/h2>\n\n\n\n<p>The dynamic inventory for Azure Cloud varies compared to GCP. For Azure, each dynamic inventory source corresponds to a single Azure subscription, effectively limiting the number of sources in the dynamic inventory. In contrast, for GCP, there&#8217;s a source for each project, which could potentially lead to a significantly large number of sources.&nbsp;<\/p>\n\n\n\n<p>For Azure, the strategy involved creating a workflow. This workflow processes the dynamic inventory&#8217;s sources, updates them, and once all have been refreshed, triggers a template to update all the constructed inventories that use this dynamic inventory as their source.&nbsp;<\/p>\n\n\n\n<p><strong>Ansible Automation Platform Templates<\/strong>&nbsp;<\/p>\n\n\n\n<p>There are two templates:&nbsp;<\/p>\n\n\n\n<ol class=\"wp-block-list\" start=\"1\">\n<li><strong>TMPL-CONSTRUCTED-INV-REFRESH-AZURE<\/strong>:&nbsp;\n<ul class=\"wp-block-list\">\n<li>Tag: azure_refresh&nbsp;<\/li>\n\n\n\n<li>Related variable: constructed_inventories_list specific to Azure<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>  - INV-CONSTRUCTED-AZURE-LINUX-ALL \n<br>\n  - INV-CONSTRUCTED-AZURE-OEL-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-RHEL-NONSAP \n<br>\n  - INV-CONSTRUCTED-AZURE-RHEL-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-SLES-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-WINDOWS \n<br>\n  - INV-CONSTRUCTED-AZURE-WINDOWS-SAP <\/code><\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"750\" src=\"https:\/\/www.insentragroup.com\/au\/wp-content\/uploads\/sites\/22\/2023\/10\/image-2-1024x750.png\" alt=\"\" class=\"wp-image-19588\" title=\"\" srcset=\"https:\/\/www.insentragroup.com\/au\/wp-content\/uploads\/sites\/22\/2023\/10\/image-2-1024x750.png 1024w, https:\/\/www.insentragroup.com\/au\/wp-content\/uploads\/sites\/22\/2023\/10\/image-2-300x220.png 300w, https:\/\/www.insentragroup.com\/au\/wp-content\/uploads\/sites\/22\/2023\/10\/image-2-768x562.png 768w, https:\/\/www.insentragroup.com\/au\/wp-content\/uploads\/sites\/22\/2023\/10\/image-2.png 1199w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>In this example, I created a new execution environment that includes the ansible.controller collection (you might need to replace it with awx.awx collection if you don\u2019t have a valid Red Hat subscription). The following snippet provides all the collections included in the execution environment:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>--- \n<br>\ncollections: \n<br>\n  - ansible.posix \n<br>\n  - community.crypto \n<br>\n  - community.general \n<br>\n  - community.podman \n<br>\n  - ansible.controller \n<br>\n  - community.hashi_vault \n<br>\n  - containers.podman <\/code><\/pre>\n\n\n\n<p>Both templates are configured within the SharedInventory organisation and utilise the specified execution environment. They draw upon the PRJ-AAP-CONSTRUCTED-INV-REFRESH project, the Demo Inventory, the site.yml playbook, and the CREDS-AAP-VAULT vault credential. Considering all tasks utilise API calls to AAP, there&#8217;s no need for a designated host to run the code. Consequently, I employ the Demo Inventory, which lists localhost as its sole host configuration.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Resync for GCP&nbsp;<\/h2>\n\n\n\n<p>As highlighted earlier, the dynamic inventory for GCP could include a large number of sources, given there&#8217;s one for each project. This extensive list makes utilising a WORKFLOW challenging. The recommended approach is to deploy Ansible code to refresh all sources within the dynamic inventory. Once this step is completed, the constructed inventory can be updated. Note that the task dedicated to refreshing GCP&#8217;s dynamic inventory is set to bypass errors. This design choice ensures that an update error in one GCP project doesn&#8217;t stop the refreshing of the constructed inventory.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Ansible Automation Platform Template&nbsp;<\/h3>\n\n\n\n<ol class=\"wp-block-list\" start=\"1\">\n<li><strong>TMPL-CONSTRUCTED-INV-REFRESH-GCP<\/strong>:&nbsp;\n<ul class=\"wp-block-list\">\n<li>Tag: gcp_refresh&nbsp;<\/li>\n\n\n\n<li>Related variable: constructed_inventories_list specific to GCP<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n\n\n\n<pre class=\"wp-block-code\"><code>constructed_inventories_list: \n<br>\n  - INV-CONSTRUCTED-GCP-RHEL \n<br>\n  - INV-CONSTRUCTED-GCP-WINDOW<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Usage<\/h2>\n\n\n\n<p>To refresh Azure constructed inventories, execute the following workflow:&nbsp;&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>TMPL-CONSTRUCTED-INV-REFRESH-AZURE-WORKFLOW <\/code><\/pre>\n\n\n\n<p>This workflow will:&nbsp;&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>resync all the Azure inventory sources in INV-SHARED-AZURE-ALL inventory<\/li>\n\n\n\n<li>resync all Azure constructed inventories&nbsp;&nbsp;<\/li>\n<\/ul>\n\n\n\n<p>To refresh GCP constructed inventories, execute the following Template:&nbsp;<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>TMPL-CONSTRUCTED-INV-REFRESH-GCP <\/code><\/pre>\n\n\n\n<p>The template will:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>resync all GCP sources in INV-SHARED-GCP inventory<\/li>\n\n\n\n<li>resync all GCP constructed inventories<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Modifications<\/h2>\n\n\n\n<p>In case additional constructed inventories are created in future, add the name of the inventory to the Azure or GCP template.&nbsp;&nbsp;<\/p>\n\n\n\n<p>For example, TMPL-CONSTRUCTED-INV-REFRESH-AZURE template has the following Extended Variables:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>--- \n<br>\nconstructed_inventories_list: \n<br>\n  - INV-CONSTRUCTED-AZURE-LINUX-ALL \n<br>\n  - INV-CONSTRUCTED-AZURE-OEL-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-RHEL-NONSAP \n<br>\n  - INV-CONSTRUCTED-AZURE-RHEL-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-SLES-SAP \n<br>\n  - INV-CONSTRUCTED-AZURE-WINDOWS \n<br>\n  - INV-CONSTRUCTED-AZURE-WINDOWS-SAP <\/code><\/pre>\n\n\n\n<p>TMPL-CONSTRUCTED-INV-REFRESH-GCP template has the following Extended Variables:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>--- \n<br>\nconstructed_inventories_list: \n<br>\n  - INV-CONSTRUCTED-GCP-RHEL \n<br>\n  - INV-CONSTRUCTED-GCP-WINDOWS \n <\/code><\/pre>\n\n\n\n<p>If you have any questions or need further details about refreshing constructed inventories using Ansible Automation Platform, feel free to <a href=\"https:\/\/www.insentragroup.com\/au\/contact\/\" target=\"_blank\" rel=\"noreferrer noopener\">contact us<\/a>!<\/p>\n\n\n\n<style>\nbody .wp-block-code>code {\n    font-family: Menlo,Consolas,monaco,monospace;\n    color: #000;\n    padding: 30px 40px;\n    border: none;\n    border-radius: 4px;\n    background: #ddd;\n}\n<\/style>\n","protected":false},"excerpt":{"rendered":"<p>Explore the advanced capabilities of Ansible Automation Platforms (AAP) Constructed Inventories empower you to manage dynamic inventories seamlessly. <\/p>\n","protected":false},"author":67,"featured_media":19590,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[19],"tags":[],"class_list":["post-19587","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-modern-workplace","entry"],"_links":{"self":[{"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/posts\/19587","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/users\/67"}],"replies":[{"embeddable":true,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/comments?post=19587"}],"version-history":[{"count":3,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/posts\/19587\/revisions"}],"predecessor-version":[{"id":22672,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/posts\/19587\/revisions\/22672"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/media\/19590"}],"wp:attachment":[{"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/media?parent=19587"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/categories?post=19587"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.insentragroup.com\/au\/wp-json\/wp\/v2\/tags?post=19587"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}