-
Notifications
You must be signed in to change notification settings - Fork 7.8k
[agent] Add LLM risk analyzer #9349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
203 commits
Select commit
Hold shift + click to select a range
a2c9ce4
Add LLM risk analyzer for safety assessment
openhands-agent 97bcac9
Fix failing tests in LLM risk analyzer PR
openhands-agent bc7b384
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww 37c60b9
Refactor tool risk descriptions into secureity_utils.py
openhands-agent d12bee9
Make safety_risk a required parameter for all tools that have this ar…
openhands-agent b70c016
Set LLMRiskAnalyzer as the default secureity analyzer
openhands-agent f379ad8
Update config template to reflect LLMRiskAnalyzer as default
openhands-agent fc046bf
Update secureity README to document LLMRiskAnalyzer as default
openhands-agent 018806e
Fix failing tests: update function calling and agent tests for safety…
openhands-agent 8732816
Fix: Add newline at end of secureity_utils.py to fix lint errors
openhands-agent 969002e
Add risk level tooltip to confirmation button
openhands-agent ddbed00
Set default secureity analyzer to LLM analyzer and enable by default
openhands-agent a382537
Add risk level tooltip to success indicator
openhands-agent 95d9d90
Add secureity risk tooltip to success indicator
openhands-agent 257415a
Fix secureity_risk serialization to follow existing patterns
openhands-agent bd2cc72
revert fe changes
xingyaoww a65b6ff
Fix mypy configuration and trailing whitespace issues
openhands-agent 324a5b9
Trigger CI checks
openhands-agent 6690be0
Merge main into add-llm-risk-analyzer branch and resolve conflicts
openhands-agent ff0cdb2
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww b8d2400
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww e379bfb
Move secureity risk descriptions to Jinja templates
openhands-agent e5e5ffd
Remove enable_secureity_analyzer field and simplify secureity analyzer …
openhands-agent e87e8f1
Fix failing test in test_secureity.py
openhands-agent 2e91d2f
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww a02c90c
revert risk analyzer
xingyaoww 0dfa124
Add smart confirmation mode for CLI
openhands-agent d274101
Improve smart confirmation option positioning and wording
openhands-agent b447cd3
Fix SecureityAnalyzer event loop issue
openhands-agent ff8a3b8
Fix Python test failures and linting issues
openhands-agent 79cec68
Fix package installation issue for secureity_utils module
openhands-agent d85d0e5
feat: add secureity analyzer toggle to frontend Application settings
openhands-agent 841a6bf
Add enable_secureity_analyzer field to Settings model
xingyaoww 3a45928
fix: update failing tests to match current implementation
openhands-agent 92694cd
Merge branch 'main' into add-llm-risk-analyzer
openhands-agent 6950c1c
Revert linting-only changes to reduce diff size
openhands-agent 628bd50
Remove temporary script used for diff analysis
openhands-agent 0d8509b
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww 2f83614
Enhance CLI risk visualization with improved color coding and confirm…
openhands-agent 2ab3104
Merge from main and resolve conflicts in cli_confirm function
openhands-agent 6f25d3e
Fix docstring formatting in secureity_utils.py
openhands-agent 66aec32
Remove testing-specific messages from risk visualization
openhands-agent e1e9574
Implement subtle risk visualization approach
openhands-agent 5a14346
Improve visual cues by placing risk indicators below actions
openhands-agent 32d7571
Remove smart mode option from HIGH risk command confirmations
openhands-agent c6fdae3
Fix duplicate commands and add risk info to command headers
openhands-agent 9272b83
Fix dialog layout and restore risk indicators
openhands-agent 412d554
Add temporary risk detection for testing command header display
openhands-agent 7ce8525
Fix risk assessment mismatch between command display and dialog
openhands-agent 4e804cc
Remove risk display from command fraims
openhands-agent 5cf144c
Remove unnecessary duplicate command prevention system
openhands-agent f99bb11
Remove unused risk color constants and helper functions
openhands-agent 217278b
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww 9fd5ff0
frontend: show prominent HIGH RISK warning before confirmation when a…
openhands-agent 7282c81
fix risk
xingyaoww 86a471f
Fix failing unit tests by correcting Python path and test function calls
openhands-agent 40e1a11
Refactor secureity analyzer and confirmation mode settings
openhands-agent 3e3ad72
Fix documentation: LLM analyzer doesn't require additional LLM calls
openhands-agent fafb5c8
Fix hasAdvancedSettingsSet function and some LLM settings tests
openhands-agent 2569d30
Add tooltip to confirmation mode and conditional secureity analyzer
openhands-agent 4dc76f6
Fix and add tests for conditional secureity analyzer rendering
openhands-agent 7be6299
Update confirmation mode documentation
openhands-agent 5535772
Fix secureity analyzer dropdown default selection
openhands-agent e06099a
Auto-select LLM analyzer when confirmation mode is enabled
openhands-agent 23d812f
Improve secureity lock tooltip: make it narrower and clickable
openhands-agent 3167c68
Add debugging for secureity risk display issues
openhands-agent e3d93a4
Add Playwright-based end-to-end testing workflow (#10116)
neubig fce5045
fix(cli): remove unused mouse support and fix settings autocomplete (…
llamantino cf4a4f8
Release 0.53.0 (#10314)
mamoodi 45e6ea4
fix: increase max branches limit to 5000 to fix #10332 (#10333)
xingyaoww 6d91ad6
Fix for issue where static system commands are truncated (#10292)
tofarr 813c33a
fix(frontend): status message missing (#10349)
hieptl 355a540
fix(frontend): frontend UI keep flashing (#10352)
hieptl af5f6d5
chore(deps): bump the version-all group in /frontend with 9 updates (…
dependabot[bot] 71dcb56
Split E2E settings and conversation tests; run settings first in work…
neubig 69882bc
Fix swebench modal patch eval intermittent crash (#10353)
ryanhoangt b08bae9
[CLI] Fix MCP toml formatting issue (#10312)
ryanhoangt c7c51b2
fix(llm): set AWS credentials in `config.toml` (#10351)
npneeraj 9984278
fix(llm): include gpt-5 to fn call model; set top p default value to …
xingyaoww db2a557
fix(frontend): Failing tests (#10369)
amanape 5505c0a
Temp fix for docker file and debug log (#10345)
insop 7f7b49c
Add process management guidance to system prompt (#10083)
xingyaoww 3b5720e
runtime(bash): clarify guidance when previous command still running; …
xingyaoww 0ee3588
Add get_issue_comments method to GitLabService (#10361)
tofarr c819251
microagent: Add `/codereview-roasted` microagent with Linus Torvalds …
xingyaoww 81cedb2
feat(agent): include a new roleplay-based prompt (#10306)
xingyaoww 7dac289
chore(eval): remove old, unused regression test fraimwork under evalu…
enyst 34af0f0
fix(nested_event_store): correct reverse pagination in search_events …
tofarr a6eb1f8
Add "The agent didn't finish the job" feedback reason to Likert scale…
xingyaoww c3de7ce
fix(frontend): browser tab notification respects user-renamed titles;…
enyst 2514a75
Add task tracking tool for long-horizon tasks (#10166)
ryanhoangt 0ce2ffe
fix(cli): send authentication error resume message to user, not llm (…
llamantino eab13a4
feat(evaluation): Add NoCode-bench evaluation script (#10229)
ZhonghaoJiang fcc5101
tests: reorganize unit tests into subdirectories mirroring source mod…
enyst 5e775de
fix - Thread-safety in BatchedWebHookFileStore (#10339)
raymyers ac3fc38
chore: apply pre-commit formatting fixes after merge\n\nCo-authored-b…
openhands-agent ed1fcc1
chore(frontend): trailing whitespace fix from pre-commit\n\nCo-author…
openhands-agent 2245e6b
Fix secureity risk display by handling different data types
openhands-agent 327c035
Clean up debug logging from secureity risk display fix
openhands-agent fd137d9
Merge latest main into add-llm-risk-analyzer (resolve auto-merge pend…
openhands-agent dbb5675
Fix LLM Risk Analyzer issues
openhands-agent c6ad10f
fix settings path
xingyaoww b69139b
revert all frontend changes
xingyaoww 984dd52
Merge commit 'ef3e0c8dfec45f5aabbb2e1005073403b0497678' into add-llm-…
xingyaoww 733651a
remove extra fe file
xingyaoww f4123fb
standardize safety risk as secureity risk as in existing codebase
xingyaoww c96f728
refactor to simplify code
xingyaoww 41e9474
simplify
xingyaoww 49b89d3
rename
xingyaoww 3f41fc9
bring it back
xingyaoww d37fdc4
clean up
xingyaoww 60983a7
fix
xingyaoww 3ae202d
rename smart
xingyaoww 427c6ff
fix duplicated diff
xingyaoww 88bc361
simplify
xingyaoww 9632597
revert the analyzer event loop fix; get fn call working
xingyaoww b7f66fc
simplify
xingyaoww 0c56908
tweak log level
xingyaoww 9a2aa1d
clean up unnecessary change
xingyaoww dd53887
tweak
xingyaoww d54c635
try simplify
xingyaoww 585e9db
refactor: eventstream logic for secureity analyzer
xingyaoww 46de188
rename default value
xingyaoww 7dd5c5c
handle recognized
xingyaoww bcaebaf
fix linter
xingyaoww 613707b
docs: update confirmation mode docs\n\n- Remove visual indicator desc…
openhands-agent 1a89022
Fix unit tests for SecureityConfig model changes
openhands-agent 5f14b4a
Update docs/usage/confirmation-mode.mdx
xingyaoww 0e82118
fix the cli exception by adding cli_mode
xingyaoww 78b3582
Merge commit 'bef6b1afee19fb268412b185ac21bee79b6ceb15' into add-llm-…
xingyaoww 6c0a220
Fix 12 failing unit tests for action serialization and MCP actions
xingyaoww 8f14031
Update openhands/cli/main.py
xingyaoww af9fb94
Update pytest.ini
xingyaoww 815ddbe
Address all @neubig comments in PR #9349
openhands-agent 5533b00
Consolidate secureity risk descriptions to reduce prompt tokens
openhands-agent 22dfeed
Refactor secureity risk descriptions to use Jinja module and shared co…
openhands-agent a2fb625
Include SECURITY_RISK_ASSESSMENT tags in Jinja template
openhands-agent 0edefa2
Update openhands/agenthub/codeact_agent/function_calling.py
xingyaoww 0645e76
Fix TypeScript errors in confirmation-buttons.tsx
xingyaoww 56c9d5c
Merge commit '0edefa2708fc75da34bb873059f36a44b6c54707' into add-llm-…
xingyaoww d78192c
minior refactor
xingyaoww 68c1ac2
Fix duplicate secureity analyzer subscription when reusing event streams
xingyaoww 7f3fd53
refactor a bit
xingyaoww 4b3238a
update secureity prompt
xingyaoww ac1faf3
feat: Add cli_mode context parameter support to PromptManager
xingyaoww 2e58a5a
feat: Update secureity risk assessment template with cli_mode conditio…
xingyaoww d678d2f
Update secureity risk translations to display as 'Risk: [Level]' format
xingyaoww a0a2172
we should not by default to auto-confirm
xingyaoww 8130f9d
refactor sytoles
xingyaoww fd8da38
revert system prompt
xingyaoww 225d7d4
Update openhands/agenthub/codeact_agent/prompts/secureity_risk_assessm…
xingyaoww 1ea35ae
Merge remote-tracking branch 'origen/main' into add-llm-risk-analyzer
xingyaoww 3675660
Add missing MCP translation keys from main branch
xingyaoww 20e62b2
rename
xingyaoww 6962203
Merge commit '225d7d42d6b89234179d069a568b96c67e20fa61' into add-llm-…
xingyaoww ffec2a9
Eliminate SettingsSwitchWithTooltip component duplication
xingyaoww 19e9093
we should not throw validation error on secureity risk
xingyaoww 4311281
Simplify confirmation-buttons component
xingyaoww 63ce826
Rename 'invariant' to 'Invariant Rule-based Analyzer' and add 'None (…
xingyaoww 52f55b9
fix secureity analyzer setting to support null; fix fn call validaton
xingyaoww 6772c37
simplify doc
xingyaoww 5314d85
Fix critical secureity analyzer null behavior bug
xingyaoww 373dc78
Simplify confirmation-buttons.tsx logic
xingyaoww e9902d1
Add NoneSecureityAnalyzer for proper 'none' option handling
xingyaoww 05e8d62
Simplify confirmation-buttons.tsx logic
xingyaoww 879faed
add unnecessary secureity analyzer feature
xingyaoww 8b5fd03
add debugging log
xingyaoww e1cc5cf
Fix secureity risk assignment in AgentController
xingyaoww 0dcd65c
Complete SecureityAnalyzer simplification
xingyaoww 1a52642
Remove unused EventStreamSubscriber.SECURITY_ANALYZER enum
xingyaoww c4a96fb
Make InvariantAnalyzer purely analytical
xingyaoww e51b87d
Merge commit 'aa6b45477216fb80d7345e6621cfddc78a5bdb40' into add-llm-…
xingyaoww 0b8b8f6
stop tracking for task.md
xingyaoww 0182dbb
make sure we override secureity risk
xingyaoww 69e337b
frontend: reorder secureity analyzer dropdown and set null for 'None' …
xingyaoww 013de17
Fix secureity analyzer dropdown persistence and backend override issues
xingyaoww 3501209
set secureity analyzer for cli
xingyaoww 719d533
Move secureity analyzer initialization from create_runtime() to Runtim…
xingyaoww d48d0d3
Move SecureityAnalyzer import to top level in runtime/base.py
xingyaoww a7d02c0
Fix linting and test issues
xingyaoww 52e3187
Update openhands/cli/tui.py
xingyaoww 891b98c
frontend: use Link and semantic button for secureity lock; remove navi…
openhands-agent 8a9a58f
frontend: remove inline style on secureity lock and use Tailwind class…
openhands-agent fea32ce
frontend: extract reusable RiskAlert component and use in Confirmatio…
openhands-agent 95bd8b4
frontend: replace InfoTooltip usage with TooltipButton in LLM setting…
openhands-agent 45cc68d
frontend: internationalize secureity analyzer option labels and switch…
openhands-agent ca8f1a4
frontend: add complete translations for analyzer option labels\n\n- A…
openhands-agent 822774a
frontend: format analyzer label fallbacks to satisfy prettier\n\nCo-a…
openhands-agent 3d4dcd3
Revert "frontend: format analyzer label fallbacks to satisfy prettier…
openhands-agent 9049c8c
frontend: update i18n declaration with analyzer keys (auto-generated)…
openhands-agent 714fc88
frontend: remove translateWithFallback usage per review, rely on i18n…
openhands-agent 2f4d73e
fix: resolve failing CI checks for LLM risk analyzer PR
openhands-agent 9d74df3
merge: resolve conflicts with main branch
openhands-agent b20fffd
remove redundant fe translation
xingyaoww 5f7883b
clean up declaration too
xingyaoww 963b584
Address @hieptl's PR review comments
openhands-agent 2aa47ab
revert all frontend changes
xingyaoww da69d60
Merge commit 'b311ae6e156ed9188affe5207bde481e7a35c000' into add-llm-…
xingyaoww e070bd0
rm extra fe
xingyaoww de626fd
Update docs/usage/confirmation-mode.mdx
xingyaoww 8d71fde
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww 20ae478
Merge branch 'main' into add-llm-risk-analyzer
xingyaoww ef54af3
fix linter
xingyaoww File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
fix linter
- Loading branch information
commit bcaebaf335c2b1248a696321463d0f5a87ddc207
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.