Refresh Manifest and Schemas November Update

[shashank-elastic]

Pull Request

Issue link(s): https://github.com/elastic/ia-trade-team/issues/744

Summary - What I changed

• Followed Schema Update steps from documentation
• Updated Schema versions latest available 9.2.0 in beats and ecs
• Download latest ECS and Beats schemas
• Downloaded latest API schemas
• Update integration manifests and schemas
  • Pre releases included for apm, pad, elastic_security, splunk
• MITRE ATT&CK mappings has new version release v18.0.0
• Kibana Issue comment updated - [Security Solution] Update MITRE ATT&CK mappings to the next version kibana#166152 (comment)

Refresh Data, Refresh Mappings and Updated Rule

❯ python -m detection_rules dev attack -h
Loaded config file: /Users/shashankks/elastic_workspace/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

Usage: detection_rules dev attack [OPTIONS] COMMAND [ARGS]...

  Commands for managing Mitre ATT&CK data and mappings.

Options:
  -h, --help  Show this message and exit.

Commands:
  refresh-data               Refresh the ATT&CK data file.
  refresh-redirect-mappings  Refresh the ATT&CK redirect file and update all rule threat mappings.
  update-rules               Update threat mappings attack data in all rules.

detection-rules on  nov_schema_refresh [$✘!+?] is 📦 v1.5.5 via 🐍 v3.12.8 (.venv) on ☁️  shashank.suryanarayana@elastic.co 
❯ python -m detection_rules dev attack refresh-data
Loaded config file: /Users/shashankks/elastic_workspace/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

No versions newer than the current detected: 18.0.0

detection-rules on  nov_schema_refresh [$✘!+?] is 📦 v1.5.5 via 🐍 v3.12.8 (.venv) on ☁️  shashank.suryanarayana@elastic.co 
❯ python -m detection_rules dev attack refresh-redirect-mappings
Loaded config file: /Users/shashankks/elastic_workspace/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

refreshing data in attack_technique_redirects.json
refreshed mapping file: /Users/shashankks/elastic_workspace/detection-rules/detection_rules/etc/attack-technique-redirects.json

detection-rules on  nov_schema_refresh [$✘!+?] is 📦 v1.5.5 via 🐍 v3.12.8 (.venv) on ☁️  shashank.suryanarayana@elastic.co took 39s 
❯ python -m detection_rules dev attack update-rules             
Loaded config file: /Users/shashankks/elastic_workspace/detection-rules/.detection-rules-cfg.json

█▀▀▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄▄▄ ▄   ▄      █▀▀▄ ▄  ▄ ▄   ▄▄▄ ▄▄▄
█  █ █▄▄  █  █▄▄ █    █   █  █ █ █▀▄ █      █▄▄▀ █  █ █   █▄▄ █▄▄
█▄▄▀ █▄▄  █  █▄▄ █▄▄  █  ▄█▄ █▄█ █ ▀▄█      █ ▀▄ █▄▄█ █▄▄ █▄▄ ▄▄█

'Delayed Execution via Ping' requires update - technique name change for tactic 'Defense Evasion'

Finished - 1 rules updated!

• Investigation Guides Required - One rule added with investigation guide.

How To Test

Checklist

• Added a label for the type of pr: bug, enhancement, schema, maintenance, Rule: New, Rule: Deprecation, Rule: Tuning, Hunt: New, or Hunt: Tuning so guidelines can be generated
• Added the meta:rapid-merge label if planning to merge within 24 hours
• Secret and sensitive material has been managed correctly
• Automated testing was updated or added to match the most common scenarios
• Documentation and comments were added for features that require explanation

Contributor checklist

• Have you signed the contributor license agreement?
• Have you followed the contributor guidelines?

[github-actions]

Enhancement - Guidelines

These guidelines serve as a reminder set of considerations when addressing adding a feature to the code.

Documentation and Context

• Describe the feature enhancement in detail (alternative solutions, description of the solution, etc.) if not already documented in an issue.
• Include additional context or screenshots.
• Ensure the enhancement includes necessary updates to the documentation and versioning.

Code Standards and Practices

• Code follows established design patterns within the repo and avoids duplication.
• Ensure that the code is modular and reusable where applicable.

Testing

• New unit tests have been added to cover the enhancement.
• Existing unit tests have been updated to reflect the changes.
• Provide evidence of testing and validating the enhancement (e.g., test logs, screenshots).
• Validate that any rules affected by the enhancement are correctly updated.
• Ensure that performance is not negatively impacted by the changes.
• Verify that any release artifacts are properly generated and tested.
• Conducted system testing, including fleet, import, and create APIs (e.g., run make test-cli, make test-remote-cli, make test-hunting-cli)

Additional Checks

• Verify that the enhancement works across all relevant environments (e.g., different OS versions).
• Confirm that the proper version label is applied to the PR patch, minor, major.

[tradebot-elastic]

⛔️ Test failed

Results

• ❌ Delayed Execution via Ping (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Test failed

Results

• ❌ Delayed Execution via Ping (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Test failed

Results

• ❌ Potential Privilege Escalation via SUID/SGID Proxy Execution (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Delayed Execution via Ping (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Test failed

Results

• ❌ Potential Privilege Escalation via SUID/SGID Proxy Execution (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Delayed Execution via Ping (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta

[tradebot-elastic]

⛔️ Test failed

Results

• ❌ Potential Privilege Escalation via SUID/SGID Proxy Execution (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta
• ❌ Delayed Execution via Ping (eql)
  • coverage_issue: no_rta
  • stack_validation_failed: no_rta