Commit graph

293 commits

Author SHA1 Message Date
Kayvan Sylvan 0dbf9cb0ec
Merge pull request #2152 from AUTHENSOR/fix/extension-executor-shell-injection 2026-07-28 07:38:30 -07:00
Kayvan Sylvan c32ee25f03 feat: add Claude Opus 5 support and refresh dependencies
## CHANGES

- Add Claude Opus 5 to supported model selection
- Disable sampling parameters for Claude Opus 5 requests
- Restrict one-million-token beta headers to compatible Claude models
- Remove unsupported 200K-context models from beta header mapping
- Upgrade Anthropic, AWS, Ollama, Google, and supporting dependencies
2026-07-24 11:47:51 -07:00
Kayvan Sylvan 72d626ebec
Merge branch 'main' into feature/upgrade-minimax-m3 2026-07-15 18:55:00 -07:00
Kayvan Sylvan 6e44e1c66a fix: respect Anthropic chat option max token overrides
- Use configured Anthropic max tokens as default
- Apply chat option max tokens when provided
- Preserve existing behavior for missing token overrides
- Add tests for default max token selection
- Add tests for explicit max token overrides
2026-07-12 11:58:36 -07:00
Kayvan Sylvan 6a999ab23e chore: cleanups - tidying the duplicate model listed 2026-07-09 13:09:50 -07:00
Kayvan Sylvan 5f34b6812e feat: add Claude Sonnet 5 Anthropic support
- Add Claude Sonnet 5 to supported Anthropic models
- Omit sampling parameters for Claude Sonnet 5 requests
- Centralize Anthropic sampling restrictions behind prefix matching
- Enable one-million-token context beta for Claude 5 models
- Remove older Claude 4 aliases from model listings
- Update Anthropic tests for Sonnet 5 beta mapping
- Refresh Go dependencies across AI provider integrations
2026-07-09 13:04:48 -07:00
Authensor 3eba7a51d4 fix: shell-escape extension values to prevent command injection
The extension executor runs commands via 'sh -c' with user-controlled
values interpolated into the command string without escaping. A value
containing shell metacharacters (;, |, $(), backticks) is executed by
the shell, enabling command injection.

User input flows from content processed through a pattern into the
extension system via the InputSentinel, then into formatCommand which
interpolates it into the cmd_template, then into exec.Command('sh', '-c').
No escaping is applied at any point.

Fix: wrap all user-controlled values (value, numbered pipe-split values)
in single quotes with embedded-single-quote escaping before interpolation.
This ensures sh -c treats them as literal arguments, not shell syntax.

The existing tests pass unchanged because the inner sh strips the single
quotes, so the executed command receives the same argument values.

Regression test added: ShellInjectionBlocked verifies that input
'hello; touch /marker' does not create the marker file.

Signed-off-by: John Kearney <johndanielkearney@gmail.com>
2026-07-01 21:55:21 -05:00
Kayvan Sylvan 59d26a7e89 feat: cache OpenAI model discovery and handle provider rate limits
- Add persistent cache for provider model discovery results
- Serve stale model caches during discovery failures
- Return concise localized errors for rate-limited model fetches
- Send GitHub Models API version header automatically
- Add Claude Fable 5 Anthropic model support
- Omit sampling parameters for Claude Fable 5
- Update model rate-limit translations across supported locales
- Add tests for cache, rate-limit, and GitHub headers
- Update Go dependencies for AI provider integrations
2026-06-09 14:36:46 -07:00
Kayvan Sylvan 4b4821a235 chore: extend sampling param exclusion to Opus 4.8 models
- Add Opus 4.8 to sampling param exclusion check
- Update comment to mention Opus 4.8 models
- Match `claude-opus-4-8` model prefix alongside 4.7
2026-06-02 16:25:48 -07:00
octo-patch be1540781d feat: upgrade MiniMax default model to M3
- Add MiniMax-M3 to the static MiniMax model list as the new default
- Retain MiniMax-M2.7 and MiniMax-M2.7-highspeed as available alternatives
- Remove deprecated older models (M2.5 / M2.5-highspeed / M2.5-lightning / M2 / M2.1 / M2.1-lightning) from the static list

MiniMax-M3 is the new flagship model and becomes the default selection by being placed first in the static model list.
2026-06-01 21:16:41 +08:00
Kayvan Sylvan 134171de7e modernize ./...
/Users/kayvan/src/fabric/internal/plugins/ai/codex/errors.go:61:5: errors.As can be simplified using AsType[*openaiapi.Error]
/Users/kayvan/src/fabric/internal/cli/flags.go:126:2: NumField/Field loop can simplified using Type.Fields iteration
/Users/kayvan/src/fabric/internal/cli/help.go:145:2: NumField/Field loop can simplified using Type.Fields iteration
/Users/kayvan/src/fabric/internal/cli/help.go:225:2: NumField/Field loop can simplified using Type.Fields iteration
/Users/kayvan/src/fabric/internal/i18n/i18n.go:222:15: strings.Split call can be simplified using strings.Cut
2026-05-28 12:27:27 -07:00
Kayvan Sylvan 4dfaf61e93 chore: bump Go toolchain and dependencies, add Claude Opus 4.8 model
# CHANGES

- Upgrade Go toolchain to 1.26.0
- Bump anthropic-sdk-go to v1.46.0
- Add Claude Opus 4.8 to supported models
- Update AWS SDK and Bedrock service modules
- Bump ollama client to v0.24.0
- Refresh OpenTelemetry, gRPC, and genai dependencies
- Update go-git, sqlite3, and assorted indirect modules
2026-05-28 12:11:30 -07:00
Kayvan Sylvan d2995ee02e fix: omit Anthropic sampling params for Claude Opus 4.7
## CHANGES

- Add Opus 4.7 sampling parameter guard
- Omit temperature and top_p for incompatible models
- Preserve existing TopP and temperature selection behavior
- Cover Opus 4.7 omission with unit test
2026-05-04 14:48:35 -07:00
Kayvan Sylvan 774376d19b
Merge branch 'main' into fix/issue-2084-vendor-model-prefix-parsing 2026-04-23 01:35:14 -07:00
Kayvan Sylvan 11f2683d50 Merge branch 'main' into feat/grokai-search-grounding 2026-04-23 00:37:28 -07:00
Kayvan Sylvan 78f9d7c1f7 fix: fall back to streamed delta text when completed Codex response is empty
- Prefer extracted completed text only when content stays non-empty
- Fall back to accumulated streamed delta text otherwise
- Preserve streamed response text before completed response evaluation
- Add regression test for empty completed output text
- Simulate SSE delta stream followed by blank completion
- Verify Send returns delta text when completion lacks content
2026-04-16 20:16:23 -07:00
Kayvan Sylvan d2537208df feat: add Claude Opus 4.7 model support and bump Anthropic SDK to v1.37.0
- Upgrade `anthropic-sdk-go` dependency from v1.34.0 to v1.37.0
- Add `claude-opus-4-7` to supported models list
- Enable 1M context window beta for Opus 4.7
- Update model beta comments to reflect Opus 4.7 support
2026-04-16 19:35:39 -07:00
alecjmckanna 0d7e1f7d52
feat: add --readpattern flag to print pattern contents to terminal
Adds a new `--readpattern <name>` CLI flag that prints the raw contents
of a named pattern's system.md file to stdout. This makes it easy to
inspect what instructions a pattern sends to the model without having
to navigate the filesystem manually.

The implementation respects custom patterns directories: it checks the
user's custom patterns directory first before falling back to the main
patterns directory, consistent with how all other pattern lookups work.
2026-04-15 00:43:16 -04:00
Kayvan Sylvan e387e7b86f refactor: replace manual reverse loops with slices.Backward iterator
- Use `slices.Backward` for reverse iteration in copilot response extraction
- Use `slices.Backward` for reverse iteration in gemini TTS extraction
- Add `slices` import to copilot and gemini packages
- Remove manual index-based reverse loop patterns
- Remove redundant comment about copilot response message type
2026-04-12 21:18:29 -07:00
Kenneth G. Hartman 7ced82f782 feat(openai): add GrokAI search grounding via xAI Responses API
xAI's Responses API accepts web_search and x_search tool types, but
fabric hardcoded OpenAI's web_search_preview tool name in
buildResponseParams, causing GrokAI plus --search to fail with HTTP 422.

This adds two new fields to openai_compatible.ProviderConfig:
- WebSearchToolName: override the default web_search tool name string
- EnableXSearch: append xAI's x_search tool when search is enabled

Both fields are empty/false by default, preserving backwards
compatibility for all existing providers. GrokAI now sets
WebSearchToolName to "web_search" and EnableXSearch to true.

Tests added in openai_test.go cover the new override paths and
confirm the default provider behavior is unchanged.

Verified with live xAI API key: fabric -V GrokAI --search "query"
now returns grounded results with real source URLs.
2026-04-11 12:27:16 -04:00
majiayu000 cb3a0e5aec
fix: parse vendor prefix from model name when vendor is not specified
When users pass a model string like "ollama/llama3", the lookup fails
because no model literally named "ollama/llama3" exists — the model is
stored as "llama3" under the "Ollama" vendor group. This adds fallback
logic to split the first path segment and check if it matches a known
vendor, resolving the "could not find vendor" error for prefixed models.

Fixes #2084

Signed-off-by: majiayu000 <1835304752@qq.com>
2026-04-08 18:17:59 +08:00
Kayvan Sylvan c78576deb7 feat: internationalize YouTube visual extraction flags and error messages
- Move visual flag descriptions to i18n locale system
- Add visual extraction strings to all 11 locale files
- Replace hardcoded English error messages with i18n lookups
- Register visual flags in `flagDescriptionMap` for help system
- Remove inline `description` tags from visual CLI flag structs
- Localize FFmpeg, Tesseract, and yt-dlp error messages
- Add `youtube_visual_frame_cue` translation key across locales
2026-04-05 15:22:28 -07:00
Kayvan Sylvan aeb709e7e8 Merge branch 'main' into feat/youtube-visual-extraction 2026-04-05 09:17:24 -07:00
Sathvik C 631d02cc27 style(youtube): remove unnecessary blank lines in GrabVisual function 2026-03-27 08:07:10 -05:00
Kayvan Sylvan c271816d2d feat: add i18n translations for Codex OAuth and error messages
- Add 17 new Codex-related i18n keys across all locale files
- Replace hardcoded English strings in OAuth flow with i18n lookups
- Internationalize Codex error messages in `errors.go`
- Localize token exchange and refresh failure messages
- Translate OAuth callback server responses and browser fallback text
- Add translations for usage limit and request status errors
- Cover DE, EN, ES, FA, FR, IT, JA, PL, PT-BR, PT-PT, ZH locales
2026-03-25 16:49:12 -07:00
Kayvan Sylvan 2e0abdd78a refactor: propagate context.Context through Vendor interface methods
- Add `context.Context` parameter to `ListModels` interface method
- Add `context.Context` parameter to `SendStream` interface method
- Thread caller context through `Chatter.Send` instead of using `context.Background()`
- Update all vendor implementations to accept context parameter
- Introduce `publicError` type wrapping Codex provider errors
- Normalize Codex error messages to lowercase for consistency
- Add context-aware `sendStreamUpdate` helper in Codex client
- Remove stale `context.Background()` calls from Anthropic and Azure AI Gateway
- Update all vendor test mocks and stubs with new signatures
- Pass HTTP request context from server handler into `chatter.Send`
2026-03-25 16:09:10 -07:00
Kayvan Sylvan 070c626b7b refactor: extract OAuth and auth logic from Codex client module
- Remove OAuth flow, PKCE, and token refresh from codex.go
- Remove auth transport round-trip retry logic
- Remove unused OAuth types and helper structs
- Remove JWT parsing and token expiry utilities
- Remove error mapping and usage limit detection helpers
- Remove browser-open and version normalization functions
- Add `.maestro/` directory to `.gitignore`
- Add test for `SendStream` HTTP error mapping and channel close
- Clean up unused imports from codex client package
2026-03-25 15:32:40 -07:00
Sathvik C f812188115 fix(youtube): resolve tesseract CLI args, racy error handling, timestamp overflow, and import ordering 2026-03-24 14:04:20 -05:00
Sathvik C 8aa7a703dc refactor(youtube): implement bounded OCR concurrency, context timeouts, and CLI argument security 2026-03-24 13:35:10 -05:00
Sathvik C e42d8b8fec feat(youtube): make visual extraction parameters configurable via CLI flags 2026-03-24 13:21:22 -05:00
Sathvik C 8368ecf35f fix(youtube): support multi-line yt-dlp outputs and modern ffmpeg syntax 2026-03-24 13:19:29 -05:00
Sathvik C 086e196ade feat(youtube): implement FFmpeg and Tesseract visual extraction 2026-03-24 12:06:57 -05:00
Kayvan Sylvan e5d0d7f630 chore: update Go module dependencies and remove deprecated Anthropic model aliases
- Bump `anthropic-sdk-go` from v1.23.0 to v1.27.1
- Upgrade AWS SDK Go v2 packages to latest patch versions
- Update `gin-gonic/gin` to v1.12.0 and `go-git` to v5.17.0
- Bump `ollama/ollama` from v0.16.2 to v0.18.2
- Upgrade `google.golang.org/api` to v0.272.0 and `genai` to v1.51.0
- Update OpenTelemetry packages to v1.42.0/v0.67.0
- Bump `golang.org/x` packages to latest minor versions
- Remove deprecated `ModelClaude4Sonnet20250514` and `ModelClaude4Opus20250514` aliases
- Add `go.mongodb.org/mongo-driver/v2` as new indirect dependency
2026-03-22 07:54:47 -07:00
Kayvan Sylvan 81aa983b03 chore: remove duplicate key in zh locale. 2026-03-19 10:55:50 -07:00
YE 71e3dcd94d Refine Chinese translations for better idiomaticity 2026-03-19 10:55:50 -07:00
Kayvan Sylvan 7023e15d0a
Merge branch 'main' into feature/upgrade-minimax-m27 2026-03-18 10:40:50 -07:00
octo-patch 42ecceab4f feat: upgrade MiniMax default model to M2.7
- Add MiniMax-M2.7 and MiniMax-M2.7-highspeed to static model list
- Place M2.7 models at the top of the list as new defaults
- Retain all previous models (M2.5, M2.5-highspeed, M2.5-lightning, M2, M2.1, M2.1-lightning) as alternatives

MiniMax-M2.7 is the latest flagship model with enhanced reasoning and coding capabilities.
2026-03-18 13:33:09 +08:00
Prax Lannister 3dd498bc47 fix(chat): prevent streaming deadlock and unify strategy handling
## Bug: Streaming Deadlock in Chatter.Send

When a streaming error occurs, two goroutines can race to write to the
same buffered(1) error channel:

1. The SendStream goroutine returns an error and writes to errChan
2. The stream-update loop receives a StreamTypeError update and also
   writes to errChan

Since errChan has a buffer of 1, the second write blocks forever,
causing a goroutine leak and a deadlock — the caller never returns.

### Fix

Introduce `recordFirstStreamError()` which uses a non-blocking
select/default to safely send only the first error, discarding
subsequent ones. This prevents the deadlock while preserving the
original error for the caller.

### Scenario that triggers the deadlock (before this fix):

  1. Vendor stream emits a StreamTypeError update (e.g., rate limit)
  2. The update loop writes the error to errChan (buffer fills)
  3. SendStream also returns an error
  4. SendStream goroutine tries to write to errChan → BLOCKS FOREVER
  5. Chatter.Send never returns → user sees a hang

## Refactor: Unify Strategy Handling (Server + CLI)

The REST API server (chat.go) was loading strategy prompts inline by
reading JSON files directly with os.ReadFile, bypassing the core
Chatter layer entirely. This caused:

- Strategy prompts were prepended to UserInput instead of the system
  message, breaking the prompt architecture
- The StrategyName was not passed to GetChatter(), so the core layer
  had no knowledge of the strategy
- Duplicate strategy-loading logic between CLI and server paths

### Fix

- Pass StrategyName through to GetChatter() and ChatRequest so the
  core Chatter.BuildSession() handles strategy loading uniformly
- Extract `buildPromptChatRequest()` helper for clean request
  construction
- Remove inline os.ReadFile strategy loading from the server handler

## Refactor: Clean System Message Assembly

Replace raw string concatenation of context + pattern + strategy
prompts with `joinPromptSections()` which:

- Trims whitespace from each section
- Skips empty sections (no double newlines from missing context)
- Joins with a single newline separator

## Tests Added

- `TestChatter_BuildSession_SeparatesSystemSections`: Verifies
  strategy, context, and pattern are joined with newline separators
  in the correct order
- `TestChatter_Send_StreamingErrorUpdateAndReturnDoesNotDeadlock`:
  Regression test with 2-second timeout that catches the deadlock
  when both error paths fire simultaneously
- `TestBuildPromptChatRequest_PreservesStrategyAndUserInput`: Verifies
  the extracted helper preserves all fields including StrategyName
2026-03-18 04:42:03 +05:30
Kayvan Sylvan 3c40a3462d feat: add OpenAI Codex vendor with browser-based OAuth authentication
- Add new Codex AI vendor plugin with OpenAI OAuth PKCE flow
- Implement browser-based login with automatic token refresh on 401
- Register Codex client in the plugin registry
- Allow explicit Codex model selection bypassing model listing
- Expose shared OpenAI `BuildResponseParams` and `ExtractText` helpers
- Move system/developer messages into Codex `instructions` field
- Add Codex i18n strings across all supported locales
- Add comprehensive unit tests for Codex OAuth, streaming, and retry
- Update README with Codex feature entry and provider listing
- Trim older changelog entries from README recent features section
2026-03-16 10:20:23 -07:00
octo-patch cb08f8e9e0 feat: add MiniMax-M2.5-highspeed to static model list
Add the MiniMax-M2.5-highspeed model variant to the static MiniMax
model list. This model offers the same performance as MiniMax-M2.5
but with faster inference speed.

API docs: https://platform.minimax.io/docs/api-reference/text-openai-api
2026-03-14 22:48:10 +08:00
Kayvan Sylvan 9057f97739
Merge pull request #2052 from praxstack/feat/bedrock-bearer-token-auth
feat(bedrock): dynamic region fetching and AWS_PROFILE conflict fix
2026-03-09 08:49:51 -07:00
Prax Lannister b211c42b44 fix: address all PR #2052 review feedback
- P2 fix: Add thread-safety comment to withMockEndpointsURL helper
- All @ksylvan reviewer comments addressed (see verification below)
2026-03-08 17:25:43 +05:30
Prax Lannister 19430ec69d Merge branch 'danielmiessler:main' into feat/bedrock-bearer-token-auth 2026-03-08 17:15:16 +05:30
pretyflaco 1c94fabb46 fix(ollama): map thinking levels and convert single system-role messages 2026-03-08 01:14:47 +03:00
Prax Lannister 2469673a2c feat(bedrock): dynamic region fetching and AWS_PROFILE fix
- Fetch Bedrock regions dynamically from botocore endpoints.json (public, no auth)
  Shows 40+ regions instead of hardcoded 6. Falls back to static list on error.
- Fix AWS_PROFILE env var conflict: users with AWS_PROFILE set for other tools
  (terraform, aws-cli) would get 'failed to get shared config profile' errors
  when using explicit ABSK or static credentials.
- Handle empty auth choice gracefully (skip instead of error)

Follow-up to #2044.
2026-03-06 23:19:31 +05:30
Kayvan Sylvan 4f9d4875a6 fix: redact API keys in config responses and eliminate shell injection surfaces
- add `maskAPIKey` to redact all but last 4 chars of API keys (CWE-200)
- add `isRedacted` guard to prevent writing masked values back to `.env`
- mask all provider API keys in `GET /config` response payload
- sanitize note filenames with `basename` and allowlist regex (CWE-78, CWE-22)
- replace `exec`/shell commands in obsidian route with native `fs` APIs
- remove `escapeShellArg` helper now that shell execution is fully eliminated
- add path-confinement double-check ensuring resolved paths stay within target dirs
- sanitize note filenames in notes route using `basename` to block path traversal (CWE-22)
- return `safeFilename` instead of raw user input in notes POST response
2026-03-06 08:53:46 -08:00
Kayvan Sylvan 24f3f82dc6 feat: internationalize Bedrock setup UI strings and add guided setup i18n keys
- Internationalize all Bedrock setup prompt strings via `i18n.T()`
- Add `bedrock_setup_*` i18n keys for auth, region, and model selection
- Add `bedrock_client_not_initialized` error message to all locale files
- Propagate new i18n keys across 11 locale files (de, en, es, fa, fr, it, ja, pl, pt-BR, pt-PT, zh)
- Replace hardcoded setup strings in `bedrock.go` with i18n lookups
- Reorder locale JSON keys alphabetically for consistency
2026-03-06 07:06:02 -08:00
Kayvan Sylvan 5d784609f0 Merge remote-tracking branch 'upstream/main' into feat/bedrock-bearer-token-auth 2026-03-05 14:04:33 -08:00
Kayvan Sylvan 1f7e1d3d20 refactor: extract raw pattern retrieval into dedicated GetRaw method`
- Add `GetRaw` method to `PatternsEntity` for unprocessed pattern retrieval
- Replace inline raw pattern loading logic in server handler with `GetRaw`
- Remove manual `Pattern` struct construction from `PatternsHandler.Get`
- Simplify server handler by delegating storage access to database layer
- Add test coverage for `GetRaw` with custom patterns directory
2026-03-05 12:19:06 -08:00
Prax Lannister e59a70f8ba feat: add Bedrock bearer token (ABSK) authentication
Add 3-tier authentication for AWS Bedrock:
1. Bearer token (ABSK) - simplest, same as Claude Code
2. Static AWS credentials (access key + secret key)
3. Default AWS credential chain (existing behavior)

- Remove hasAWSCredentials() gate so Bedrock always appears in setup
- Custom guided Setup() flow: auth method, region, model
- Bearer token transport with token redaction
- Nil guards for uninitialized clients
- Temperature-only fix (no top_p) for Claude on Bedrock
- API key masking during re-setup
- Fallback model list when ListFoundationModels API is inaccessible
- i18n support for 11 locales (3 new keys each)
- 25 unit tests
2026-03-05 11:30:55 +05:30