This plugin relies heavily on a feature within NetBox known as Staged Changes, which, while still experimental, is a crucial component for the proper functioning of the plugin. We’ve found significant success using this feature to address various use cases. For a more comprehensive explanation of Staged Changes, please refer to the official NetBox documentation.
If the ingestion job is configured to automatically merge staged changes, the plugin will automatically merge the changes into the NetBox database once the branch reaches the
Staged Changes in NetBox
Staged Changes is a feature in NetBox that provides a programmatic API for staging the creation, modification, and deletion of objects within the system. Importantly, these changes are not immediately committed to the active database. This feature is especially valuable for scenarios like conducting “dry runs” of bulk operations or preparing a set of changes for administrative review before they are applied.
Branches: A Conceptual Overview
In the context of this plugin, we introduce the concept of branches, which operates in a manner analogous to Git branches. A branch in this context represents a segregated workspace within NetBox where changes are isolated from the live database until explicitly merged.
Key Functions of Branches
Isolation of Changes
Just like Git branches, any modifications made within a branch remain separate from the live database until the user decides to merge those changes. This isolation ensures that database integrity is maintained during the change proposal and review process.
Proposal and Review of Changes
Our plugin leverages branch functionality to propose changes to the NetBox database. These proposed changes can be reviewed by users to assess their potential impact before committing them to the live database. Users have the option to selectively merge these changes based on their preferences.
Within our plugin, we introduce a custom data model called
IPFabricBranch. This custom model is a subclass of the built-in
Branch model in NetBox, enhanced with additional fields to suit the specific requirements of our plugin.
Additional Fields in
sync field is implemented as a
ForeignKey reference to the
IPFabricSync model. It serves the purpose of tracking the origin of a branch, indicating the Ingestion/Sync operation from which the branch was created.
job field is also implemented as a
ForeignKey reference, but it points to the NetBox
Job model. This field is used to associate a background job with a branch, providing visibility into the job responsible for executing tasks within the branch.
By combining the power of Staged Changes and the concept of branches, our plugin offers a controlled and structured approach to making changes to the NetBox database. Users can propose, review, and selectively merge these changes, promoting efficient and transparent database management.
During the synchronization process, a crucial step is the creation of a branch. The branch detail page offers valuable insights into this process, providing users with information about various aspects of the ingestion. This includes details about the associated Ingestion object, the related job, and more. If the branch is still in progress, the page automatically updates to display real-time information about the synchronization process, including status updates and logs.
Here’s an overview of the key information and functionalities available on the Branch Detail page:
1. Branch Information
- Branch Name: Displays the name of the branch, providing a unique identifier for tracking.
2. Ingestion Object
- Ingestion: Links to the associated Ingestion object, providing context about the synchronization task.
3. Associated Job
- Job: Links to the related job, enabling users to access detailed information about the job’s progress and status.
4. Real-Time Updates
- Status Updates: The page continuously updates to reflect the current status of the synchronization process. This real-time feedback helps users monitor progress.
- Logs: Users can access logs related to the branch and synchronization process. These logs provide information about each step of the process, aiding in troubleshooting and diagnostics. In future these logs will be expanded to contain more detail.
This page offers transparency into the synchronization process, making it easier for users to monitor progress, access logs, and ensure the successful execution of data ingestion tasks.
Viewing Changes in a Completed Branch
Once a branch reaches the
completed state, users can review the proposed changes that will be applied to the NetBox database by clicking the
Changes tab. This section provides a clear overview of the changes that are ready for merging. The displayed information is presented in a table format with the following columns:
- Name: Represents the name of the object or action associated with the change.
- Action: Specifies the type of action to be performed, such as create, update, or delete.
- Object Type: Identifies the type of object being modified or created.
- Diff: Offers a detailed view of the differences (diff) between the proposed changes and the current state of the object.
Diff column provides additional information depending on the type of action:
- Creating an Object: When creating an object, the diff section displays a serialized version of the object, providing a snapshot of the new object’s state.
- Updating an Object: When updating an object, the diff section highlights the specific changes that will be applied to the object, making it easy to identify what modifications will occur.
Here are a couple of examples illustrating the
In this example, creating an object shows the serialized version of the new object being added during the ingestion process.
Here, updating an object involves altering the hostname through a transform map. The diff section reflects the changes that will be applied to the object.
Additionally, custom fields are updated for the device model to track the last sync time for a device.
Merging a Branch
To accept and integrate the branch into the NetBox database, users can utilize the
Merge button located in the top right corner. This action commits the proposed changes to the NetBox database and clears all staged changes, ensuring that the synchronized data is accurately reflected in the system.
Merge functionality simplifies the process of applying changes from a completed branch to the NetBox database, streamlining data synchronization and maintaining data integrity.