Skip to main content

Discovery Patterns

Discovery patterns enable you to create custom discovery rules for specialized infrastructure, applications, and services. Using pattern-based discovery, NopeSight can identify and classify complex systems that standard discovery methods might miss or misinterpret.

Pattern Architecture​

Pattern Processing Flow​

Pattern Structure​

Pattern Definition:
  id: pattern_oracle_rac
  name: Oracle RAC Cluster
  version: 1.2
  author: DBA Team
  description: Identifies Oracle Real Application Clusters
  
  triggers:
    - type: process
      match: "ora_pmon_.*"
    - type: port
      value: 1521
    - type: file
      path: "/etc/oratab"
      
  identification:
    confidence_threshold: 85
    require_all: false
    min_matches: 2
    
  enrichment:
    - extract_version
    - identify_nodes
    - map_dependencies
    - check_licensing

Creating Discovery Patterns​

Pattern Types​

Application Patterns

pattern:
  name: "SAP System"
  type: "application"
  
  detection:
    processes:
      - name_regex: "disp\+work|sapstart|jstart"
      - command_contains: "pf=/sapmnt"
    
    ports:
      - range: "3200-3299"  # SAP Dispatcher
      - range: "3300-3399"  # SAP Gateway
      
    files:
      - path: "/sapmnt/*/profile/DEFAULT.PFL"
      - path: "/usr/sap/*/SYS/profile/*"
      
  classification:
    ci_type: "Business Application"
    subtype: "SAP System"
    attributes:
      - sap_sid: extract_from_path
      - instance_number: extract_from_process
      - system_type: detect_from_profile

Infrastructure Patterns

pattern:
  name: "Kubernetes Node"
  type: "infrastructure"
  
  detection:
    processes:
      - name: "kubelet"
      - name: "kube-proxy"
      
    files:
      - path: "/etc/kubernetes/kubelet.conf"
      - path: "/var/lib/kubelet/config.yaml"
      
    network:
      - port: 10250  # Kubelet API
      - port: 10255  # Kubelet read-only
      
  enrichment:
    api_calls:
      - endpoint: "https://localhost:10250/pods"
        auth: "bearer_token"
        extract:
          - pod_count
          - namespaces
          - container_runtime

Database Patterns

pattern:
  name: "PostgreSQL Cluster"
  type: "database"
  
  detection:
    processes:
      - name: "postgres"
        args_contains: "-D"
      
    ports:
      - 5432  # Default PostgreSQL
      - range: "5433-5440"  # Common alternatives
      
    files:
      - path: "*/postgresql.conf"
      - path: "*/pg_hba.conf"
      
  advanced_detection:
    sql_query: |
      SELECT version(), 
             current_setting('server_version'),
             pg_is_in_recovery()
      
  classification:
    determine_role:
      - if: "pg_is_in_recovery = true"
        then: "replica"
      - else: "primary"

Pattern Syntax​

Basic Matchers

# String matching
exact_match: "nginx"
contains: "apache"
starts_with: "java -jar"
ends_with: ".exe"
regex: "^python[0-9]+(\\.[0-9]+)?"

# Numeric matching
equals: 8080
greater_than: 1024
less_than: 65535
range: "8000-9000"
in_list: [80, 443, 8080, 8443]

# File matching
exists: "/etc/nginx/nginx.conf"
contains_text: "server_name example.com"
size_greater: "1MB"
modified_within: "7d"

Complex Conditions

conditions:
  - and:
    - process: "java"
    - port: 8080
    - or:
      - file: "/opt/tomcat/conf/server.xml"
      - file: "/usr/local/tomcat/conf/server.xml"
      
  - custom_script: |
      if (hasProcess("oracle") && hasPort(1521)) {
        version = extractFromProcess(/ORACLE_HOME=([^\\s]+)/);
        return {
          match: true,
          metadata: { oracle_home: version }
        };
      }

Pattern Development​

Pattern Builder UI​

Visual Pattern Builder:
  1. Drag-and-drop conditions
  2. Test against live data
  3. Preview matches
  4. Validate logic
  5. Export as YAML/JSON

Features:
  - Syntax highlighting
  - Auto-completion
  - Pattern validation
  - Test data sets
  - Version control

Pattern Testing​

// Test pattern against sample data
const testData = {
  hostname: "app-server-01",
  processes: [
    { name: "java", cmdline: "-jar app.jar -Xmx2g" },
    { name: "nginx", cmdline: "nginx: master process" }
  ],
  ports: [80, 443, 8080],
  files: [
    { path: "/etc/nginx/nginx.conf", exists: true },
    { path: "/opt/app/config.yml", exists: true }
  ]
};

const pattern = loadPattern("custom_app_stack");
const result = pattern.evaluate(testData);

console.log({
  matched: result.matched,
  confidence: result.confidence,
  extracted_data: result.metadata
});

Pattern Validation​

Validation Rules:
  Syntax:
    - Valid YAML/JSON
    - Required fields present
    - Proper data types
    - Regex compilation
    
  Logic:
    - No infinite loops
    - Reasonable thresholds
    - Valid API endpoints
    - Executable scripts
    
  Performance:
    - Execution time < 5s
    - Memory usage < 50MB
    - API calls < 10
    - File reads < 20

Advanced Patterns​

Multi-Stage Discovery​

pattern:
  name: "Microservices Application"
  type: "composite"
  
  stages:
    1_identify_containers:
      detect:
        - process: "dockerd"
        - file: "/var/run/docker.sock"
      action: list_containers
      
    2_analyze_containers:
      foreach: container
      detect:
        - image_contains: "myapp/"
        - env_var: "SERVICE_NAME"
      action: extract_metadata
      
    3_map_services:
      correlate:
        - container_labels
        - network_connections
        - environment_variables
      build: service_topology
      
    4_create_cis:
      foreach: service
      create:
        type: "Microservice"
        relationships: auto_detected

AI-Assisted Patterns​

pattern:
  name: "Unknown Application"
  type: "ai_discovery"
  
  initial_detection:
    unusual_ports: true
    unknown_processes: true
    custom_paths: true
    
  ai_analysis:
    model: "application_classifier"
    features:
      - process_names
      - port_numbers
      - file_paths
      - network_behavior
      
    output:
      application_type: predicted
      confidence: percentage
      similar_to: known_patterns
      
  human_review:
    if_confidence_below: 70
    notify: discovery_team
    actions:
      - approve_classification
      - create_new_pattern
      - mark_for_investigation

Dependency Patterns​

pattern:
  name: "Web Application Stack"
  type: "dependency_map"
  
  components:
    load_balancer:
      identify:
        - process: ["nginx", "haproxy"]
        - config_contains: "upstream"
        
    web_servers:
      identify:
        - connection_from: load_balancer
        - process: ["apache", "nginx", "iis"]
        
    app_servers:
      identify:
        - connection_from: web_servers
        - process: ["java", "python", "node"]
        
    databases:
      identify:
        - connection_from: app_servers
        - port: [3306, 5432, 1521, 27017]
        
  relationships:
    - from: load_balancer
      to: web_servers
      type: "distributes_to"
      
    - from: web_servers
      to: app_servers
      type: "forwards_to"
      
    - from: app_servers
      to: databases
      type: "queries"

Pattern Library​

Built-in Patterns​

Categories:
  Operating Systems:
    - Windows Server Detection
    - Linux Distribution Identification
    - Container OS Recognition
    
  Databases:
    - Oracle RAC Clusters
    - SQL Server Always On
    - MongoDB Replica Sets
    - PostgreSQL Streaming
    
  Web Servers:
    - Apache/Nginx Configuration
    - IIS Application Pools
    - Tomcat Instances
    
  Applications:
    - SAP Systems
    - Microsoft Exchange
    - Active Directory
    - Kubernetes Clusters
    
  Cloud Services:
    - AWS Service Detection
    - Azure Resource Identification
    - GCP Component Discovery

Community Patterns​

Pattern Sharing:
  Repository: https://patterns.nopesight.community
  
  Categories:
    - Verified: Tested by NopeSight
    - Community: User contributed
    - Beta: Under development
    
  Contribution Process:
    1. Fork pattern repository
    2. Add pattern file
    3. Include test data
    4. Document usage
    5. Submit pull request
    
  Quality Metrics:
    - Downloads
    - Success rate
    - False positive rate
    - Performance impact

Pattern Management​

# CLI pattern management
# List available patterns
nopesight pattern list --category database

# Import pattern
nopesight pattern import --file oracle_rac.yaml --validate

# Test pattern
nopesight pattern test oracle_rac --target 10.1.1.50

# Enable/disable pattern
nopesight pattern enable oracle_rac --scope production
nopesight pattern disable legacy_app --reason "deprecated"

# Export patterns
nopesight pattern export --format yaml --output patterns_backup.zip

Pattern Optimization​

Performance Tuning​

Optimization Strategies:
  Order Conditions:
    - Most selective first
    - Cheap operations before expensive
    - Common cases optimized
    
  Cache Results:
    - Pattern match results
    - Extracted metadata
    - API responses
    
  Batch Operations:
    - Group similar patterns
    - Combine API calls
    - Bulk file reads
    
  Early Termination:
    - Stop on first match (if appropriate)
    - Skip remaining conditions
    - Fast fail on mismatches

Pattern Metrics​

{
  "pattern_performance": {
    "oracle_rac": {
      "executions": 1523,
      "matches": 47,
      "avg_duration": "2.3s",
      "success_rate": "98.2%",
      "false_positives": 2,
      "last_updated": "2024-01-10"
    },
    "kubernetes_node": {
      "executions": 3891,
      "matches": 234,
      "avg_duration": "1.1s",
      "success_rate": "99.7%",
      "false_positives": 0,
      "last_updated": "2024-01-14"
    }
  }
}

Troubleshooting Patterns​

Common Issues​

Pattern Not Matching

Diagnostic Steps:
  1. Enable debug logging
  2. Check individual conditions
  3. Verify data availability
  4. Test regex patterns
  5. Review confidence threshold

Debug Output:
  Pattern: oracle_database
  Target: 10.1.1.50
  Conditions:
    ✓ Process 'oracle' found
    ✗ Port 1521 not open
    ✓ File /etc/oratab exists
  Result: No match (2/3 conditions)

Performance Issues

Symptoms:
  - Slow pattern evaluation
  - High CPU usage
  - Memory consumption

Solutions:
  - Optimize regex patterns
  - Reduce API calls
  - Implement caching
  - Adjust parallelism
  - Simplify conditions

Pattern Debugging​

# Enable pattern debugging
import nopesight

# Set debug level
nopesight.patterns.debug_level = "TRACE"

# Test specific pattern
pattern = nopesight.patterns.load("complex_app")
pattern.trace_evaluation = True

result = pattern.evaluate(target_host)

# Analyze trace
for step in result.trace:
    print(f"{step.timestamp}: {step.condition} = {step.result}")
    if step.error:
        print(f"  Error: {step.error}")

Best Practices​

1. Pattern Design​

  • ✅ Start simple, add complexity
  • ✅ Use meaningful names
  • ✅ Document thoroughly
  • ✅ Include examples

2. Testing​

  • ✅ Test with real data
  • ✅ Verify edge cases
  • ✅ Check false positives
  • ✅ Measure performance

3. Maintenance​

  • ✅ Version control patterns
  • ✅ Regular reviews
  • ✅ Update for changes
  • ✅ Deprecate obsolete

4. Sharing​

  • ✅ Sanitize sensitive data
  • ✅ Include test cases
  • ✅ Document prerequisites
  • ✅ Share with community

Next Steps​