# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Repository Overview

This is the **Reporting Templates** repository for NetFoundry's Monitoring Operations Platform (MOP). It contains Elasticsearch search templates (mustache) used for querying and aggregating network metrics and event data across multiple deployment environments (sandbox, staging, production).

The repository is organized as a Terraform module that manages Elasticsearch query templates as infrastructure. Each template is a parameterized Elasticsearch query that filters and aggregates data based on network topology, performance metrics, and connectivity attributes.

**Current Active Branch**: `feature/MOP-27467-reporting-templates-add-cc-search-criteria-for-ncfabriccircuits`
**Current Work**: Adding Customer Connect (CC) search criteria support to fabric circuit reporting templates

## Technology Stack

- **Infrastructure as Code**: Terraform (v1.1.7+)
- **Query Language**: Mustache templating for Elasticsearch queries
- **Providers**:
  - `elasticstack` (v~0.9) - for managing Elasticsearch query templates
  - `aws` (v~3.0) - for authentication and state management
- **Backend**: Terraform state stored in AWS S3 with DynamoDB locking
- **CI/CD**: Jenkins (Groovy-based pipeline)

## Project Structure

```
reporting-templates/
├── terraform/                          # All Terraform configurations
│   ├── main.tf                        # Provider configurations (AWS, Elasticsearch)
│   ├── versions.tf                    # Required provider versions
│   ├── variables.tf                   # Variables (ES_USR, ES_PSW credentials)
│   ├── locals.tf                      # Workspace-specific configuration
│   ├── *.tf                           # Template definitions grouped by feature
│   └── .terraform.lock.hcl            # Terraform lock file
├── jenkinsfiles/
│   ├── serverless-deploy.jenkinsfile  # Main deployment pipeline (sandbox → staging)
│   └── serverless-deploy-prod.jenkinsfile  # Production deployment
├── bck_reference/                     # Backup/reference copies of previous template versions
└── .gitignore                         # Standard Terraform ignores

```

## Template Organization

Templates are organized by functional area. Each `.tf` file contains related Elasticsearch query templates:

| File | Purpose | Key Templates |
|------|---------|---|
| `ncfabriccircuits*.tf` | Fabric circuit dial logs and health | service_endpoint_dial_logs, service_dial_health, identity_traffic_analysis |
| `ncutilization*.tf` | Network utilization metrics (RX/TX, component usage) | rx_and_tx_dashboard, top_components, metrics_usage_sum |
| `nclinklatency*.tf` | Link latency measurements and timeouts | fabric_latency, timeouts data queries |
| `ncctrllatency*.tf` | Controller latency metrics | controller_latency filtering and aggregation |
| `ncevents*.tf` | Network events filtering and aggregation | filter_criteria, date_time_range, endpoint_last_seen |
| `ncserviceevents*.tf` | Service health events | health_count, health_time_series |
| `ipsec_tunneler*.tf` | IPsec tunnel metrics | bridge_health, events, metrics_timeseries |
| `ncentityevent*.tf` | Entity change events | all_events, dashboard_events |
| `metricbeat.tf` | Metricbeat monitoring agent queries | CPU/memory usage for controllers/fabric |
| `componentusage.tf` | Component-level usage aggregations | |

### Versioning Convention

Many templates exist in two versions:
- **Base version** (e.g., `ncfabriccircuits.tf`) - Uses older field names (nf_endpoint_id, nf_service_id, @timestamp)
- **v8 version** (e.g., `ncfabriccircuits_v8.tf`) - Uses newer field naming (identity_id, service_id, timestamp) with additional Customer Connect filtering

The `_v8` templates are the newer standard for new deployments.

## Template Structure & Patterns

All templates follow this Terraform + Elasticsearch pattern:

```hcl
resource "elasticstack_elasticsearch_script" "template_name" {
    script_id = "template_name"
    lang      = "mustache"  # Uses Mustache templating syntax
    source    = <<-EOT
    { 
      # Elasticsearch query DSL with Mustache variables like {{networkId}}, {{gte}}, {{lte}}
      # Conditional filters using {{#variableName}} ... {{/variableName}}
    }
    EOT
}
```

**Critical**: Always use `<<-EOT` heredoc for `source`. Never use `jsonencode({...})` — Terraform's HCL parser will fail on any `{{mustache}}` blocks outside string values.

### Common Mustache Variables

Templates use Mustache templating to accept dynamic parameters:

**Date/Time**:
- `{{gte}}` - greater than or equal to timestamp
- `{{lte}}` - less than or equal to timestamp
- `{{interval}}` - time bucket interval for aggregations
- `{{calendar_interval}}` / `{{aggInterval}}` - calendar-based intervals (1d, 1h, etc.)
- `{{timeZone}}` - timezone for date histogram aggregations

**Network Identifiers**:
- `{{networkId}}` - Network identifier (required in most templates)
- `{{networkGroupId}}` - Network group identifier (required in most templates)

**Entity Filters**:
- `{{endpointId}}` / `{{identity_id}}` - Endpoint/identity identifier
- `{{endpointName}}` / `{{identity_name}}` - Endpoint/identity name
- `{{serviceId}}` / `{{service_id}}` - Service identifier
- `{{serviceName}}` / `{{service_name}}` - Service name
- `{{hostName}}` / `{{host_name}}` - Host/router name

**Customer Connect (CC) Filters** (new - added in current feature branch):
- `{{ccHostLocation}}` - Host location list (terms filter)
- `{{ccClientLocation}}` - Client location list (terms filter)
- `{{ccHostConnector}}` - Host connector list (terms filter)
- `{{ccClientConnector}}` - Client connector list (terms filter)
- `{{ccHostCustomer}}` - Host customer identifier list (terms filter)
- `{{ccClientCustomer}}` - Client customer identifier list (terms filter)

**Other Parameters**:
- `{{size}}` - Number of documents to return
- `{{aggTermSize}}` - Number of terms for aggregations
- `{{attributes}}` - Array of attribute values (terms filter)
- `{{latencyType}}` - Which latency field to measure (avg, p95, p99)

### Conditional Filtering with Mustache

Filters are optional using Mustache conditionals:
```json
{{#endpointId}},
{
  "match_phrase": {
    "identity_id.keyword": "{{endpointId}}"
  }
}
{{/endpointId}}
```

If `endpointId` is not provided/empty, the entire block is omitted from the query.

## Deployment & Operations

### Terraform Workspaces

The project uses Terraform workspaces for environment isolation:
- `nf-sandbox` → sandbox Elasticsearch
- `nf-staging-int` → staging Elasticsearch  
- `nf-prod-int` → production Elasticsearch

Workspace configuration is in `locals.tf` with environment-specific Elasticsearch URLs and IAM roles.

### Deployment Pipeline

**Sandbox/Staging (Automated)**:
```
Jenkins: serverless-deploy.jenkinsfile
  Initialize → Deploy to sandbox → Deploy to staging
  Compares current git hash against deployed hash (stored in AWS SSM)
  Plans changes, applies Terraform
  Records new hash in SSM parameter store
```

**Production (Manual)**:
```
Jenkins: serverless-deploy-prod.jenkinsfile
  Similar process but for production workspace
  Requires manual trigger
```

### Terraform Commands

```bash
# Select workspace and prepare
terraform workspace select nf-sandbox
terraform init

# View what will change
terraform plan

# Apply changes
terraform apply -auto-approve

# List all workspaces
terraform workspace list
```

### Credentials

Elasticsearch credentials are passed via environment variables:
- `TF_VAR_ES_USR` - Elasticsearch username
- `TF_VAR_ES_PSW` - Elasticsearch password

In Jenkins, these are sourced from credentials store: `jenkins_template_reader_{environment}`

## Development Workflow

### Making Changes to Templates

1. **Select a template file** based on the feature area (see Template Organization table above)

2. **Modify the Elasticsearch query** within the `source` attribute:
   - Keep the overall query structure (bool query with must/filter clauses)
   - Add new filter conditions in the `filter` array for new search criteria
   - Use conditional Mustache blocks `{{#variableName}}` for optional parameters
   - Use `"terms"` filter for multi-value list parameters (e.g., Customer Connect filters)

3. **Test locally** (if Elasticsearch access available):
   ```bash
   cd terraform
   terraform plan -var ES_USR=user -var ES_PSW=pass
   ```

4. **Follow existing patterns**:
   - Use `{{#toJson}}variableName{{/toJson}}` for array parameters in terms filters
   - Use `strict_date_optional_time||epoch_millis` format for timestamp ranges
   - Keep indentation consistent with existing templates
   - Document template purpose in header comments

5. **Versioning**:
   - If modifying v8 templates (preferred), update the v8 version
   - Consider whether the change should apply to base versions too
   - Do NOT modify base versions without understanding v8 impact

### Common Modifications

**Adding a new search filter**:
```json
{{#myNewFilter}},
{
  "match_phrase": {
    "field.keyword": "{{myNewFilter}}"
  }
}
{{/myNewFilter}}
```

**Adding a new list filter** (like CC filters):
```json
{{#myList}},
{
  "terms": {
    "field.keyword": {{#toJson}}myList{{/toJson}}
  }
}
{{/myList}}
```

**Changing timestamp field names**:
- Base templates use `@timestamp`
- v8 templates use `timestamp`
- Keep format as: `"format": "strict_date_optional_time||epoch_millis"`

### Git Workflow

The repository follows feature branch naming: `feature/MOP-{ticket}-{description}`

```bash
# Review changes before committing
git diff
git status

# Typical commit messages follow the pattern:
# "add support for [feature]" or "[action] [description]"

# Push to remote when ready for Jenkins deployment
git push
```

## Common Development Tasks

### Adding Customer Connect Filtering to a Template

The current feature branch adds CC support to templates. Pattern to follow:

```json
// In the filter array, add:
{{#ccHostLocation}},
{
  "terms": {
    "customer_connect.host.location.keyword": {{#toJson}}ccHostLocation{{/toJson}}
  }
}
{{/ccHostLocation}}
{{#ccClientLocation}},
{
  "terms": {
    "customer_connect.client.location.keyword": {{#toJson}}ccClientLocation{{/toJson}}
  }
}
{{/ccClientLocation}}
{{#ccHostConnector}},
{
  "terms": {
    "customer_connect.host.connector.keyword": {{#toJson}}ccHostConnector{{/toJson}}
  }
}
{{/ccHostConnector}}
{{#ccClientConnector}},
{
  "terms": {
    "customer_connect.client.connector.keyword": {{#toJson}}ccClientConnector{{/toJson}}
  }
}
{{/ccClientConnector}}
{{#ccHostCustomer}},
{
  "terms": {
    "customer_connect.host.customer.keyword": {{#toJson}}ccHostCustomer{{/toJson}}
  }
}
{{/ccHostCustomer}}
{{#ccClientCustomer}},
{
  "terms": {
    "customer_connect.client.customer.keyword": {{#toJson}}ccClientCustomer{{/toJson}}
  }
}
{{/ccClientCustomer}}
```

### Reviewing Template Changes

When reviewing diffs, check:
1. Field naming is consistent with template version (v8 vs base)
2. Mustache conditional syntax is correct (opening `{{#` and closing `{{/`)
3. JSON syntax is valid within the query
4. New filters don't break existing ones (commas before new filter blocks in arrays)
5. Elasticsearch DSL follows correct nested structure (bool → must/filter)

## Key Considerations

1. **Template Scope**: These templates are read-only query patterns. They don't create, modify, or delete data—only filter and aggregate existing metrics.

2. **Field Naming Matters**: The v8 templates expect v8-indexed data with different field names. Mixing versions will cause query failures.

3. **Multi-value Filters**: Customer Connect fields accept arrays/lists; use `terms` filters with `{{#toJson}}` for proper serialization.

4. **Date Format**: Always use `strict_date_optional_time||epoch_millis` to support both ISO 8601 timestamps and epoch milliseconds.

5. **Aggregation Size**: `size` parameters for terms aggregations default to 10; set `{{aggTermSize}}` in aggregation configs or explicitly in the template.

6. **Backward Compatibility**: Base templates must continue working; new features go into v8 templates first, then backported if needed.

## Useful References

- **Elasticsearch Query DSL**: Query structure follows standard Elasticsearch bool queries
- **Mustache Documentation**: Templates use standard Mustache syntax for variables and conditionals
- **Terraform ElasticStack Provider**: https://registry.terraform.io/providers/elastic/elasticstack/latest
- **Jira Tickets**: MOP-27467 (current CC support work), MOP-27454 (recent query optimizations)