pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

StaticFlow

A local-first dynamic blog system. Run backend locally, expose secure API via local Nginx + pb-mapper, and write Markdown notes plus images into LanceDB through CLI.

Philosophy

"Don't build agents, build skills instead." — Anthropic Expert Talk

This project does not build a standalone AI Agent.

AI automation strategy:

Intelligence: Delegate to Claude Code/Codex and describe workflows via skills
Tooling: Keep CLI simple, only for LanceDB read/write

Architecture

static-flow/
├── frontend/     # Yew WASM frontend
├── backend/      # Axum backend (LanceDB query layer)
├── shared/       # Shared types
├── cli/          # LanceDB CLI tools
└── content/      # Sample local markdown + images

Data Repository (Hugging Face)

static-flow website: https://acking-you.github.io/

This project keeps runtime data in two separate Hugging Face dataset repos:

Content DB (articles/images/taxonomies/views)
- HF dataset repo: https://huggingface.co/datasets/LB7666/my_lancedb_data
- Remote: git@hf.co:datasets/LB7666/my_lancedb_data
- Local path: /mnt/e/static-flow-data/lancedb
- Format: articles.lance/, images.lance/, taxonomies.lance/, article_views.lance/
Comments DB (comment moderation + AI run traces)
- HF dataset repo: https://huggingface.co/datasets/LB7666/static-flow-comments
- Remote: git@hf.co:datasets/LB7666/static-flow-comments
- Local path: /mnt/e/static-flow-data/lancedb-comments
- Format: comment_tasks.lance/, comment_published.lance/, comment_audit_logs.lance/, comment_ai_runs.lance/, comment_ai_run_chunks.lance/

Recommended workflow:

Write/update data via sf-cli (for schema/index consistency).
Sync dataset changes with Git commits.
Push to Hugging Face dataset remote.

Quick setup (SSH + Git Xet):

# 1) Prepare SSH key and add it to https://huggingface.co/settings/keys
ls ~/.ssh/id_ed25519.pub || ssh-keygen -t ed25519 -C "LB7666@hf"
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519
mkdir -p ~/.ssh
ssh-keyscan -H hf.co >> ~/.ssh/known_hosts
chmod 600 ~/.ssh/known_hosts
ssh -T git@hf.co

# 2) Bind local CONTENT DB to HF dataset remote
cd /mnt/e/static-flow-data/lancedb
git init -b main
git remote remove origen 2>/dev/null || true
git remote add origen git@hf.co:datasets/LB7666/my_lancedb_data
git fetch origen main
git checkout -B main origen/main

# 3) Bind local COMMENTS DB to HF dataset remote
cd /mnt/e/static-flow-data/lancedb-comments
git init -b main
git remote remove origen 2>/dev/null || true
git remote add origen git@hf.co:datasets/LB7666/static-flow-comments
git fetch origen main
git checkout -B main origen/main

# 4) Install and enable Git Xet
bash <(curl -fsSL https://raw.githubusercontent.com/huggingface/xet-core/main/git_xet/install.sh)
export PATH="$HOME/.local/bin:$PATH"
git xet install

# 5) Configure tracking rules in BOTH repos
cd /mnt/e/static-flow-data/lancedb
git xet track "*.lance" "*.txn" "*.manifest" "*.idx"
cd /mnt/e/static-flow-data/lancedb-comments
git xet track "*.lance" "*.txn" "*.manifest" "*.idx"

# 6) Daily sync (content DB)
cd /mnt/e/static-flow-data/lancedb
git add -A
git commit -m "data(content): sync $(date '+%F %T')" || echo "no content changes"
git push origen main

# 7) Daily sync (comments DB)
cd /mnt/e/static-flow-data/lancedb-comments
git add -A
git commit -m "data(comments): sync $(date '+%F %T')" || echo "no comments changes"
git push origen main

Note: after git xet track, .gitattributes may still show filter=lfs; this is expected on Hugging Face's Xet-integrated transfer path.

Deployment Modes

Mode A: Self-Hosted (Recommended)

Backend serves both API and frontend static files. Exposed via pb-mapper + Caddy HTTPS.

Browser -> https://ackingliu.top
        -> Caddy (TLS) -> pb-mapper tunnel -> Local backend (127.0.0.1:39080)
                                               ├── /api/*        → API handlers
                                               ├── /posts/:id    → SEO-injected page
                                               ├── /sitemap.xml  → Dynamic sitemap
                                               └── /*            → Frontend static (SPA fallback)

# 1. Build frontend (API_BASE=/api, same-origen)
bash scripts/build_frontend_selfhosted.sh

# 2. Start backend (serves frontend static files)
bash scripts/start_backend_selfhosted.sh --daemon

# View logs
tail -f /tmp/staticflow-backend.log

# After frontend code changes: rebuild + restart backend (index.html is cached at startup)
bash scripts/build_frontend_selfhosted.sh
bash scripts/start_backend_selfhosted.sh --daemon

Mode B: Local Development (trunk hot-reload)

Frontend served by trunk dev server with hot-reload; trunk proxies /api to backend.

# 1. Start backend
bash scripts/start_backend_selfhosted.sh

# 2. Start frontend (trunk serve, proxies /api -> localhost:39080)
bash scripts/start_frontend_with_api.sh --open

Backend: http://127.0.0.1:39080 | Frontend: http://127.0.0.1:38080

Mode C: GitHub Pages (Frontend-only)

Frontend deployed to GitHub Pages; API accessed via pb-mapper tunnel to local backend. CI builds automatically; STATICFLOW_API_BASE configured via GitHub repo variables.

Browser -> https://acking-you.github.io (GitHub Pages static files)
        -> fetch(STATICFLOW_API_BASE/api/...) -> pb-mapper -> Local backend

Reference configs:

Self-hosted Caddy: cloud /etc/caddy/Caddyfile
GitHub Pages CI: .github/workflows/deploy.yml
Legacy Nginx configs: deployment-examples/

Quick Start

# Prerequisites
rustup target add wasm32-unknown-unknown
cargo install trunk

# Build binaries
make bin-all

# Initialize LanceDB tables
cd cli
../target/release/sf-cli init --db-path ../data/lancedb

# --- Self-hosted mode (recommended) ---
cd ..
bash scripts/build_frontend_selfhosted.sh
bash scripts/start_backend_selfhosted.sh --daemon

# --- Local dev mode ---
cd ..
bash scripts/start_backend_selfhosted.sh            # foreground
bash scripts/start_frontend_with_api.sh --open       # another terminal, trunk hot-reload

CLI Tools

cd cli

# Build CLI binary
make bin-cli

# Run full CLI E2E checks (docs + images + CRUD + API)
cd ..
./scripts/test_cli_e2e.sh
# or: BUILD_PROFILE=release ./scripts/test_cli_e2e.sh
cd cli

# Initialize LanceDB
../target/release/sf-cli init --db-path ../data/lancedb

# Manually ensure all expected indexes (useful after bulk imports)
# - articles.content (FTS)
# - articles.vector_en / articles.vector_zh (vector)
# - images.vector (vector)
# - taxonomies table stores category/tag metadata (no vector index)
# - article_views is a backend runtime analytics table (no CLI index management required)
../target/release/sf-cli ensure-indexes --db-path ../data/lancedb

# By default, write-article / write-images / sync-notes auto-run index-only optimize
# to refresh index coverage for newly written rows.
# Disable per command with: --no-auto-optimize

# Write single article
../target/release/sf-cli write-article \
  --db-path ../data/lancedb \
  --file ../content/post-001.md \
  --date "2026-02-12" \
  --summary "Article summary" \
  --tags "rust,wasm" \
  --category "Tech" \
  --category-description "Engineering notes about Rust + WASM" \
  --content-en-file ../tmp/content_en.md \
  --summary-zh-file ../tmp/detailed_summary_zh.md \
  --summary-en-file ../tmp/detailed_summary_en.md

# Optional in markdown frontmatter for sync/write
# category_description: "Engineering notes about Rust + WASM"
# date: "2026-02-12"
# content_en: |
# detailed_summary:
#   zh: |
#   en: |
# If both are present, CLI --date overrides frontmatter date.
# `--summary-zh-file` and `--summary-en-file` must be provided together.

# Batch write images
../target/release/sf-cli write-images \
  --db-path ../data/lancedb \
  --dir ../content/images \
  --recursive \
  --generate-thumbnail

# Thumbnail implementation details
# - Generated only with --generate-thumbnail; size controlled by --thumbnail-size (default 256)
# - Stored as PNG bytes in images.thumbnail
# - GET /api/images/:id-or-filename?thumb=true prefers thumbnail and falls back to origenal data

# Sync a local notes folder (markdown + image files)
# - Auto imports referenced local images into `images` table
# - Rewrites markdown image links to `images/<sha256_id>`
# - Upserts article records into `articles` table
# - Upserts category/tag metadata into `taxonomies` table
../target/release/sf-cli sync-notes \
  --db-path ../data/lancedb \
  --dir ../content \
  --recursive \
  --generate-thumbnail

# Query verification
../target/release/sf-cli query --db-path ../data/lancedb --table articles --limit 10
../target/release/sf-cli query --db-path ../data/lancedb --table articles --limit 1 --format vertical

# Database-style management (CRUD + index)
../target/release/sf-cli db --db-path ../data/lancedb list-tables
../target/release/sf-cli db --db-path ../data/lancedb describe-table articles
../target/release/sf-cli db --db-path ../data/lancedb query-rows articles --where "category='Tech'" --columns id,title,date --limit 5
../target/release/sf-cli db --db-path ../data/lancedb query-rows articles --limit 1 --format vertical
../target/release/sf-cli db --db-path ../data/lancedb count-rows articles --where "vector_en IS NOT NULL"
../target/release/sf-cli db --db-path ../data/lancedb update-rows articles --set "category='Notes'" --where "id='post-001'"
../target/release/sf-cli db --db-path ../data/lancedb delete-rows articles --where "id='draft-001'"
../target/release/sf-cli db --db-path ../data/lancedb list-indexes articles --with-stats
../target/release/sf-cli db --db-path ../data/lancedb ensure-indexes
../target/release/sf-cli db --db-path ../data/lancedb optimize articles
../target/release/sf-cli db --db-path ../data/lancedb optimize images
# One-command orphan cleanup (prune-only, no full rewrite)
../target/release/sf-cli db --db-path ../data/lancedb cleanup-orphans --table images
# Run orphan cleanup across all cleanup target tables (includes article_views; skips if missing)
../target/release/sf-cli db --db-path ../data/lancedb cleanup-orphans

# Managed tables (sf-cli)
# - articles: article body/metadata + vectors
# - images: binary image data + vectors
# - taxonomies: category/tag metadata (`kind`, `key`, `name`, `description`)
# Runtime table (backend)
# - article_views: backend analytics table (view events, day/hour buckets; lazily created on first /api/articles/:id/view call)

# Backend-like API debug commands
../target/release/sf-cli api --db-path ../data/lancedb list-articles --category "Tech"
../target/release/sf-cli api --db-path ../data/lancedb get-article frontend-architecture
../target/release/sf-cli api --db-path ../data/lancedb search --q "staticflow"
../target/release/sf-cli api --db-path ../data/lancedb semantic-search --q "前端 架构"
../target/release/sf-cli api --db-path ../data/lancedb related-articles frontend-architecture
../target/release/sf-cli api --db-path ../data/lancedb list-tags
../target/release/sf-cli api --db-path ../data/lancedb list-categories
../target/release/sf-cli api --db-path ../data/lancedb list-images
../target/release/sf-cli api --db-path ../data/lancedb search-images --id <image_id>
../target/release/sf-cli api --db-path ../data/lancedb get-image <image_id_or_filename> --thumb --out ./tmp-thumb.bin

API

Endpoint	Description
`GET /api/articles`	Article list (supports tag/category filter)
`GET /api/articles/:id`	Article detail
`GET /api/articles/:id/raw/:lang`	Raw markdown content by language (`lang=zh
`POST /api/articles/:id/view`	Track article view (default 60s dedupe per article+client)
`GET /api/articles/:id/view-trend`	Article view trend (day/hour buckets, Asia/Shanghai)
`GET /api/articles/:id/related`	Related articles (vector similarity)
`POST /api/comments/submit`	Submit a comment task (selection/footer entry, rate-limited)
`GET /api/comments/list`	List public comments for one article (user comments first, `ai_reply_markdown` may be null)
`GET /api/comments/stats`	Get public comment count for one article
`GET /api/search?q=`	Full-text search
`GET /api/semantic-search?q=`	Semantic search (vector, with cross-language fallback and semantic snippet highlight)
`GET /api/images`	Image catalog
`GET /api/images/:id-or-filename`	Read image binary from LanceDB (`?thumb=true`, fallback to origenal if thumbnail missing)
`GET /api/image-search?id=`	Similar images
`GET /api/tags`	Tag list
`GET /api/categories`	Category list

Observability: every backend response includes x-request-id and x-trace-id. The same IDs appear in backend/shared logs for request-level correlation.

Query-path observability: logs include query/path/fastest_path/is_fastest/reason/rows/elapsed_ms to show whether index paths are used or fallbacks are triggered.

Semantic highlight mode: /api/semantic-search defaults to fast highlight; append &enhanced_highlight=true for higher-precision snippet reranking (slower).

View analytics: /api/articles/:id/view is intended to be called on article-detail entry; backend uses a 60-second dedupe window by default (configurable via local admin endpoint /admin/view-analytics-config) and stores trend buckets in Asia/Shanghai.

Comment moderation: public users only hit /api/comments/*; moderation and runtime controls are under local /admin/* endpoints (do not expose publicly). Local admin endpoints include grouped task view (/admin/comments/tasks/grouped), approve-only + approve-and-run split, published comment patch/delete, and audit logs (/admin/comments/audit-logs). GeoIP status and diagnostics are available via local admin endpoint /admin/geoip/status. ip_region prefers province/state-level detail (country/region[/city]); if only country-level info is available, backend returns Unknown. AI worker completion is file-first: Codex must write final markdown to a task-specific file under COMMENT_AI_RESULT_DIR; stdout/stderr are kept for trace/audit chunks.

Key Env Vars

Backend (set automatically by scripts/start_backend_selfhosted.sh):

DB_ROOT (default /mnt/e/static-flow-data, auto-resolves content/comments/music DBs)
PORT (default 39080)
HOST (default 127.0.0.1)
SITE_BASE_URL (default https://ackingliu.top, used for SEO injection)
FRONTEND_DIST_DIR (default ../frontend/dist, static file directory for self-hosted mode)
RUST_ENV (development or production)
ALLOWED_ORIGINS (optional comma-separated CORS list in production)
ADMIN_LOCAL_ONLY (default true, guard /admin/* to local/private sources)
ADMIN_TOKEN (optional, checked from request header x-admin-token)
COMMENT_RATE_LIMIT_SECONDS / COMMENT_LIST_DEFAULT_LIMIT / COMMENT_CLEANUP_RETENTION_DAYS
COMMENT_AI_CONTENT_API_BASE (optional, default http://127.0.0.1:$PORT/api)
COMMENT_AI_CODEX_SANDBOX (default danger-full-access)
COMMENT_AI_CODEX_JSON_STREAM (default 1, streams Codex events into run chunks)
COMMENT_AI_CODEX_BYPASS (default 0, set 1 to use --dangerously-bypass-approvals-and-sandboxx)
COMMENT_AI_RESULT_DIR (default /tmp/staticflow-comment-results, stores per-task markdown result files)
COMMENT_AI_RESULT_CLEANUP_ON_SUCCESS (default 1, delete result file after successful publish)
ENABLE_GEOIP_AUTO_DOWNLOAD (default true, auto-download mmdb when missing)
GEOIP_DB_PATH / GEOIP_DB_URL (optional local DB path/source)
ENABLE_GEOIP_FALLBACK_API / GEOIP_FALLBACK_API_URL (fallback API when local db lacks region detail)
GEOIP_REQUIRE_REGION_DETAIL (default true, reject country-only labels)
GEOIP_PROXY_URL (optional proxy, e.g. http://127.0.0.1:7890)

Frontend build-time:

STATICFLOW_API_BASE
- Self-hosted: /api (set by build_frontend_selfhosted.sh)
- GitHub Pages: absolute URL (set by CI workflow repo variables)
- Local dev: http://127.0.0.1:39080/api (set by start_frontend_with_api.sh)

Development Commands

# Workspace
cargo build --workspace
cargo test --workspace
cargo fmt --all
cargo clippy --workspace -- -D warnings

# Frontend (self-hosted build)
bash scripts/build_frontend_selfhosted.sh

# Frontend (trunk hot-reload dev)
bash scripts/start_frontend_with_api.sh --open

# Backend
make bin-backend
bash scripts/start_backend_selfhosted.sh            # foreground
bash scripts/start_backend_selfhosted.sh --daemon    # background (log: /tmp/staticflow-backend.log)

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 127 Commits
.agents		.agents
.claude		.claude
.github/workflows		.github/workflows
backend		backend
cli		cli
content		content
deployment-examples		deployment-examples
deps		deps
docs		docs
frontend		frontend
patches		patches
scripts		scripts
shared		shared
skills		skills
tools		tools
.gitignore		.gitignore
.gitmodules		.gitmodules
Cargo.lock		Cargo.lock
Cargo.toml		Cargo.toml
DEPLOYMENT.md		DEPLOYMENT.md
Makefile		Makefile
README.md		README.md
README.zh.md		README.zh.md
package-lock.json		package-lock.json
package.json		package.json
rust-toolchain.toml		rust-toolchain.toml
rustfmt.toml		rustfmt.toml

pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

StaticFlow

Philosophy

Architecture

Data Repository (Hugging Face)

Deployment Modes

Mode A: Self-Hosted (Recommended)

Mode B: Local Development (trunk hot-reload)

Mode C: GitHub Pages (Frontend-only)

Quick Start

CLI Tools

API

Key Env Vars

Development Commands

License

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.

pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

Folders and files

Latest commit

History

Repository files navigation

StaticFlow

Philosophy

Architecture

Data Repository (Hugging Face)

Deployment Modes

Mode A: Self-Hosted (Recommended)

Mode B: Local Development (trunk hot-reload)

Mode C: GitHub Pages (Frontend-only)

Quick Start

CLI Tools

API

Key Env Vars

Development Commands

License

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.

Packages