Architectural Models

Transforming Raw Assets into a Rich Dependency Graph

2. Model Engine: Building Your Digital Twin (data/models/*.toml)

While Assets provide the raw data, the Model Engine is where you define the architecture. Models are declarative blueprints in TOML that transform asset data into a rich, contextual dependency graph. Each model file, located in data/models/, typically introduces a new type of resource by deriving it from an existing one.

File Header

The model engine provides a rich context of data that you can access within your templates, allowing for highly dynamic configurations. This data comes from three primary sources defined in the model file’s header.

# data/models/server.toml
# 1. The `origin_resource`: The primary data source for iteration.
#    This model will run once for every "application" resource.
origin_resource = "application"

# 2. Inline TOML Data: Static data defined directly in the model.
#    Useful for constants or configuration maps.
server_specs = { standard_cpu = 4, large_cpu = 8 }

# 3. External JSON Files: Data loaded from an external file.
#    Ideal for larger datasets managed separately.
network_data = { "json!" = "data/networks.json" }

This data is then accessible within templates, for example: {{ origin_resource.name }}, {{ server_specs.standard_cpu }}, or {{ network_data.subnets.prod }}.

Additionally, these custom data keys are evaluated as templates themselves and are available throughout the entire file. This allows you to create variables that depend on origin_resource or other custom variables, simplifying complex logic.

# In data/models/server.toml
origin_resource = "application"
server_specs = { standard_cpu = 4, large_cpu = 8 }

# A variable that uses another variable from the header and origin_resource
cpu_type = "{% if origin_resource.environment == 'prod' %}large_cpu{% else %}standard_cpu{% endif %}"

[[create_resource]]
resource_type = "server"
# ...
[create_resource.properties]
# The `cpu_type` variable is resolved first for each application,
# then used to look up a value in the `server_specs` map.
cpu_cores = "{{ server_specs[cpu_type] }}"

Creating Derived Resources: [[create_resource]]

This is the primary directive for creating new resources. It transforms a single origin_resource into one or more new, derived resources, forming the backbone of your architectural model. [[create_resource]] is versatile and supports several distinct patterns for generating your graph:

1. Direct Derivation: Creating a new resource based on the existence of an origin_resource (e.g., a server for every application). This can be unconditional or made conditional with match_on.
2. Data-Driven Derivation: Creating new resources based on the data inside a property of an origin_resource (using create_from_property). This is a powerful convention-over-configuration pattern for creating multiple resources from a list or string.

The following sections explain each pattern in detail.

Unconditional Creation

This rule creates one new resource for every origin_resource.

Goal: For every application, create a corresponding server resource.

# The model reads resources of type "application".
origin_resource = "application"

server_specs = { standard_cpu = 4, large_cpu = 8 }

[[create_resource]]
resource_type = "server"
relation_type = "HOSTED_ON"
name = "{{origin_resource.name}}_server"
[create_resource.properties]
os = "Linux"
cpu_cores = "{{ server_specs.standard_cpu }}" # Access data from header
managed_by = "{{origin_resource.owner}}"   # Inherit property from application

The [[create_resource]] block has several powerful options:

relation_properties: You can also add properties to the newly created relationship using a [create_resource.relation_properties] block.

```toml
[[create_resource]]
resource_type = "server"
relation_type = "HOSTED_ON"
name = "{{origin_resource.name}}_server"
[create_resource.properties]
# ... server properties
[create_resource.relation_properties]
source_app_env = "{{ origin_resource.environment }}"
managed = true
```

• Result: An application named billing-api results in a new server named billing-api_server. (application:billing-api) -[HOSTED_ON]-> (server:billing-api_server)

This process can be visualized as follows:

Conditional Creation with match_on

Goal: For every application running in the prod environment, create a corresponding classification resource to label it as restricted.

# In data/models/classification.toml
origin_resource = "application"

[[create_resource]]
match_on = [
  { property = "environment", value = "prod" }
]
resource_type = "classification"
relation_type = "IS_CLASSIFIED_AS"
name = "{{origin_resource.name}}_prod"
[create_resource.properties]
level = "Restricted"

• Result: An application with environment: "prod" will be linked to a new classification resource named <app_name>_prod with level: "Restricted". Applications in other environments will be ignored by this rule.

The match_on block supports several operators:

To express OR logic, use a nested or block:

origin_resource = "server"

match_on = [
  { property = "status", value = "active" }, # Must be active AND...
  { or = [ [ { property = "cores", greater = 8 } ], [ { property = "legacy_system", value = "false" } ] ] } # (...have >8 cores OR not be a legacy system)
]

Convention-Over-Configuration with create_from_property

This directive creates new resources by convention, using a property’s name as the new resource’s type and its value as the primary key. If the property contains an array or comma-separated string, it creates a resource for each item.

Goal: From an application’s service property, create service resources.

data/assets/application.csv

name,service
zabbix,"monitoring, alerting"

data/models/application_services.toml

origin_resource = "application"

[[create_resource]]
create_from_property = "service"
relation_type = "PROVIDES"
# You can optionally customize the name and properties
name = "svc-{{ property.value | upper }}" # `property.value` refers to the item being processed
[create_resource.properties]
category = "Operations"

• Result: Two service resources are created (svc-MONITORING, svc-ALERTING) and linked to the zabbix application via a PROVIDES relationship.

The logic of create_from_property is a powerful convention:

1. It reads the property specified (e.g., service).
2. The name of the property (service) becomes the type of the new resource(s).
3. It checks if the property’s value is a list (an array or a comma-separated string).
4. For each item in the list, it uses the item’s value ("monitoring") as the primary key for the new resource. This can be customized with the name template, where {{ property.value }} refers to the item being processed.
5. It creates the specified relationship from the origin resource to each new resource.

Deriving from a Related Node

You can also derive resources from a property on a related node using property_origin and control the new relationship’s starting point with relation_origin.

Goal: From a subscription, find its related application, read the service property, and create a service resource linked back to the original subscription.

# In data/models/subscription_services.toml
origin_resource = "subscription"

[[create_resource]]
property_origin = "application"     # 1. Look on the related 'application' node.
create_from_property = "service"    # 2. Use its 'service' property.
relation_type = "USES"              # 3. The new relation's type.
relation_origin = "origin_resource" # 4. Start the new relation from the 'subscription'.

• Result: A USES relationship is created from the subscription to the service. If relation_origin were omitted, the relationship would be created from the application.

Propagating and Linking Data

Comparison of Creation Patterns

[[copy_property]]: Propagating Data Between Connected Nodes

The [[copy_property]] directive is a declarative tool for propagating data between resources that are already directly connected. It “pushes” one or more property values from a source node to a destination node, serving as the primary mechanism for ensuring contextual data flows through your graph along existing relationships.

Think of it as inheriting properties. For example, a server resource might need to inherit the environment (prod, dev) from the application it hosts. copy_property makes this a simple, declarative rule rather than a complex join.

It supports three main patterns: unconditional propagation, conditional propagation, and renaming properties.

Unconditional Propagation (Push)

This is the most common use case: pushing properties from the origin_resource to all connected resources of a specific type.

Goal: For every application, propagate its environment and owner properties to all of its connected server resources.

# In a model with origin_resource = "application"
origin_resource = "application"

[[copy_property]]
to = "server"
properties = ["environment", "owner"]

• Graph Impact: If an application named billing-api has environment: "prod" and is connected to two servers, server-1 and server-2, this rule will add the environment: "prod" property to both server resources.

Conditional Propagation with match_on

You can use a match_on block to apply the copy operation selectively, only affecting destination nodes that meet certain criteria.

Goal: Only copy an application’s high-priority sla property to its server resources that are marked as critical.

origin_resource = "application"
[[copy_property]]
to = "server"
match_on = [ { property = "status", value = "critical" } ]
properties = [
  "network",                                      # Copies the 'network' property with the same name.
  { from = "owner", as = "subscription_owner" }   # Copies 'owner' and renames it.
]

[[link_resources]]: Enriching Data from Across the Graph

This powerful directive performs a “join” across the graph to enrich a resource. It finds another, potentially unrelated, resource based on matching property values and then performs actions like creating a relationship or copying properties.

Action: copy_properties

This is used to “pull” data from another resource. A common pattern is to enrich a resource with data from a central “lookup” table, like a location.

Goal: For every subscription, find the location resource that matches its location property and copy the compliance framework from it.

Graph Impact: An application with sla: "gold" is connected to server-1 (status: "critical") and server-2 (status: "normal"). The sla property will be copied only to server-1.

Renaming Properties During Copy

To avoid naming collisions or to provide more specific context, you can rename a property as it’s copied using the { from = "...", as = "..." } syntax.

Goal: Copy the owner property from an application to its server, but rename it to server_owner on the destination.

origin_resource = "application"

[[copy_property]]
to = "server"
properties = [
  "environment",  # Copy 'environment' with the same name
  { from = "owner", as = "server_owner" } # Copy and rename 'owner'
]

• Graph Impact: The server resource will gain a server_owner property with the value from the application’s owner property.

Mutating Properties During Copy with template

In addition to renaming, you can transform a property’s value during the copy operation using a template. The original value from the source property is available within the template as {{ value }}. This is a powerful feature for normalizing or extracting data as it flows through the graph.

Goal: Copy the version from an application, but only store the major version number on the server.

origin_resource = "application"

[[copy_property]]
to = "server"
properties = [
  # Assuming application.version is "2.7.1"
  { from = "version", as = "major_version", template = "{{ value | split(pat='.') | first }}" }
]

• Graph Impact: A server connected to an application with version: "2.7.1" will gain a major_version property with the value "2".

When to use copy_property vs. link_resources

While both directives can add properties to a resource, they serve fundamentally different purposes.

# In data/models/subscription.toml
origin_resource = "subscription"

[[link_resources]]
with = "location" # The type of the "remote" resource to join with.
join = { local = "location", remote = "name" } # The join condition.
copy_properties = [ "compliance" ]

• Graph Impact: For a subscription with location: "fra", this rule finds the location resource with name: "fra" and copies its compliance property to the subscription.

In short:

• Use copy_property to move data along paths that already exist.
• Use link_resources to find data across the graph and, in the process, create new paths or enrich nodes from distant sources.

Action: create_relation

You can also use link_resources to create a new relationship between the joined resources.

• join performs a property-based join, similar to a SQL JOIN ... ON a.column = b.column. It creates a link between two specific resources if and only if their designated properties have the same value.
• match_on / match_with perform filtered set linking. They define two sets of resources based on property filters and then create links between all members of the first set and all members of the second set. This is a set-based operation, not a value-based join.
• No on or match* performs an unconditional “singleton” join, linking all origin_resource nodes to a single, globally unique resource type.

These three patterns are complementary, not competing. Let’s examine each one to illustrate their distinct roles.

The join Statement: Precise Property-Based Joins

The join statement is the most precise way to link resources. Its purpose is to model direct, key-based relationships, analogous to a foreign key in a relational database.

Mechanism: The join clause takes a table with local and remote keys. It instructs the importer to create a relationship between an origin_resource (local) and a with resource (remote) if the value of the local property on the origin resource is identical to the value of the remote property on the remote resource.

Primary Use Case: Use join when you need to connect specific resources based on a shared identifier. This is the correct choice for modeling dependencies where one resource explicitly references another by a key.

origin_resource = "server"

[[link_resources]]
with = "package"
join = { local = "application", remote = "application" } # or short form join = "application"
create_relation = { type = "DEPENDS_ON" }

[[link_resources]]
with = "monitoring_agent"
# The server refers to its agent by 'agent_id', but the agent's primary key is 'id'.
join = { local = "agent_id", remote = "id" }
create_relation = { type = "HAS_AGENT" }

Analysis:

• First Block: This rule iterates through every server resource. For each system, it reads the value of its application property (e.g., “zabbix”). It then searches for a package resource that also has an application property with the exact same value (“zabbix”). If a match is found, a DEPENDS_ON relationship is created between that specific system and that specific package.
• Second Block: This rule iterate through every server resource, but joins agent_id and monitoring_agent resources based on server.audit_id = monitoring_agent.id property value.

This is a classic, precise join. It would be impossible to replicate this logic efficiently with match_on/match_with, which do not compare property values between resources.

match_on and match_with: Filtered Set Linking

These keywords are used to define relationships between groups of resources, not individual pairs based on a shared key.

Mechanism:

1. origin_resource defines the initial set of source nodes.
2. match_on filters this source set to create a smaller, targeted group.
3. with defines the initial set of remote nodes.
4. match_with filters this remote set to create a targeted group.
5. A relationship is created from every node in the filtered source group to every node in the filtered remote group. This is effectively a cross-product of the two filtered sets.

Primary Use Case: Use match_on and match_with for applying broad, policy-like connections. For example, “all applications in the ‘operation’ domain should be tracked by the central inventory system” or “this specific application needs a link to that specific code registry.”

Example from the code (system/link_registry.toml):

origin_resource = "package"

[[link_resources]]
match_on = [{ property = "name", value = "rescile" }]
with = "registry"
create_relation = { type = "HAS_REPOSITORY" }
match_with = [{ property = "function", value = "code" }]

Analysis:

• This rule finds a specific subset of package resources: only those where name is “rescile” (match_on).
• It also finds a specific subset of registry resources: only those where function is “code” (match_with).
• It then creates a HAS_REPOSITORY link between the found ‘rescile’ package and the found ‘code’ registry.
• Crucially, it does not compare any properties between the package and the registry. It simply links the two filtered sets.

The Unconditional “Singleton” Join

This is a special case of link_resources where you want to connect many resources to a single, central resource type.

Mechanism: Simply omit the join, match_on, and match_with clauses. The importer will create a relationship from every instance of the origin_resource to the resource(s) of the with type. This is most effective when the with resource type is a singleton (i.e., there is only one instance of it in the graph).

Primary Use Case: Use this to link all components of an environment to a central anchor, such as a tenant, subscription, or resident object.

Example from the code (subscription.toml):

# Link system resources to the subscription to establish service usage.
origin_resource = "system"
[[link_resources]]
with = "subscription"
create_relation = { type = "DEPENDS_ON" }

# Link domain resources to the subscription to establish service usage.
origin_resource = "domain"
[[link_resources]]
with = "subscription"
create_relation = { type = "BASELINE_TEMPLATE" }

Analysis:

• First Block: This rule finds every single system resource in the graph and creates a DEPENDS_ON relationship from each one to the subscription resource.
• Second Block: This does the same for every domain resource.
• This is a highly efficient way to establish a central point of reference for all resources in the model without needing any shared keys.

Comparison Table

View these as three distinct tools in your modeling toolbox, each with a clear and separate purpose:

1. Use join for specific, value-based joins.
2. Use match_on / match_with for broad, filter-based set linking.
3. Use the unconditional syntax for linking to a central singleton resource.

Automatic Relationship Creation for Derived Resources

After all architectural models ([[create_resource]], etc.) have been processed, rescile performs an automatic relationship creation phase. This is similar to the initial asset-to-asset relationship creation but targets resources created by models.

The convention is as follows:

1. The importer identifies all resource types that were created by model files (e.g., server, service, classification).
2. It then scans all resources in the graph (both from assets and other models).
3. If a resource has a property whose key matches the name of a model-defined resource type, the importer treats the property’s value as a primary key.
4. It then creates a relationship from the resource to the corresponding model-defined resource.

This allows for a powerful, convention-based way to link resources without writing explicit rules for every connection.

Example: Linking a host asset to a model-defined server

• A model file creates server resources from application resources. A server named billing-api_server is created.

• data/assets/host.csv defines physical hosts and has a server column.

  # data/assets/host.csv
  name,ip_address,server
  host-01,10.0.1.10,billing-api_server
  

During this phase, the importer sees that host-01 has a property server. Since server is a known model-defined resource type, it creates a relationship: (host:host-01) -[server]-> (server:billing-api_server).

Preventing Automatic Linking

In some cases, a property name might coincidentally match a resource type, leading to an unwanted relationship. To prevent a property from being used for automatic linking, prefix its name with an underscore (_) in your model file’s [create_resource.properties] block or in your asset CSV header. The importer will create the property on the resource without the leading underscore but will exclude it from the auto-linking process.

[create_resource.properties]
# The `service` property will be created on the new resource, but `rescile`
# will not attempt to create a relationship from it to a `service` resource.
_service = "{{ origin_resource.service_name }}"

Templating with Tera

All string values in name, properties, and match_on are processed by the Tera templating engine, enabling dynamic logic.

Accessing Data

You can access several types of data within your templates:

• origin_resource: An object containing all properties of the resource being processed.
• origin_resource_counter: A zero-based counter that increments for each origin_resource processed by a [[create_resource]] rule. It’s useful for creating unique, numbered resources.
• counter(key) function: A stateful, keyed counter for advanced numbering scenarios. For example, counter(key=origin_resource.environment) would maintain a separate count for “prod”, “dev”, etc., which is more flexible than the basic origin_resource_counter.
• Custom Data: Any inline TOML tables or external JSON data defined in the file header.
• Related Resources: When traversing a relationship (e.g., origin_resource.database[0]), the target resource’s properties are available, along with a special _relation object containing the properties of the connecting edge.
• Network Filters: Specialized filters for network calculations, such as cidr_split_n(n=4) to divide a network into subnets. See Advanced Modeling for more details.

You can also perform more complex data transformations, such as collecting attributes from related nodes.

[create_resource.properties]
resource_index = "{{ origin_resource_counter }}" # Simple index for this rule
env_index = "{{ counter(key=origin_resource.environment) }}" # Keyed index, e.g., 0 for 'prod', 0 for 'dev', 1 for 'prod', etc.
hostname = "{{ origin_resource.name | upper | replace(from='.', to='-') }}" # Chained filters
sla_level = "{% if origin_resource.environment == 'prod' %}Gold{% else %}Silver{% endif %}" # Conditional logic
db_connection_type = "{{ origin_resource.database[0]._relation.label }}" # Access edge properties
# Collect all 'volume' attributes from related 'application' nodes into a JSON string.
volumes = "{{ origin_resource.application | map(attribute='volume') | json_encode }}"

For more details on advanced templating, see the Advanced Modeling documentation.