diff --git a/.gitignore b/.gitignore index 20b5d90..0e9aa1a 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,6 @@ node_modules/ .DS_Store +.tmp/ dist/ coverage/ .env diff --git a/README.md b/README.md index a61cda1..d8e11b2 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,131 @@ -# Embargo +# Myst -Your code, under embargo. +> My stuff, kept private. -Embargo is a companion service for Forgejo that lets repo owners create expiring, revocable, read-only share links for private code snapshots. +Myst is a companion service for Forgejo that lets repo owners create expiring, revocable, read-only share links for code snapshots. -## Phase 1 PR Breakdown +## Architecture -Phase 1 is split into these reviewable branches: +```mermaid +flowchart TB + subgraph VPS["VPS / Self-Hosted"] + subgraph Docker["Docker Network"] + Forgejo["Forgejo (Git Server)"] + Postgres["PostgreSQL (2 Databases)"] + Myst["Myst (This Service)"] + Traefik["Traefik (Reverse Proxy)"] + end + Tailscale["Tailscale (Admin Access)"] + end -1. `codex/phase1-bootstrap-monorepo` -2. `codex/phase1-cli-init-skeleton` -3. `codex/phase1-service-foundation` -4. `codex/phase1-forgejo-integration` -5. `codex/phase1-grants-viewer` -6. `codex/phase1-dashboard-logs-settings` + subgraph External["External"] + Owner["Repo Owner"] + Viewer["Link Recipient"] + end -## Core Principles + Owner -->|"Login via Forgejo Session"| Myst + Owner -->|"Create View Link (PAT)"| Forgejo + Myst -->|"Read-only API (PAT)"| Forgejo + Myst -->|"Store Grants"| Postgres + Forgejo -->|"Sessions"| Postgres + Myst -->|"Public Viewer"| Viewer + Myst -->|"Admin UI (Protected)"| Owner + Tailscale -->|"Private Admin Access"| Myst + Traefik -->|"Route HTTPS Traffic"| Myst +``` -- Embargo runs beside Forgejo and talks to it only through the Forgejo API. -- Share links pin to an immutable `commit_sha`, not a moving branch. -- Embargo uses its own owner auth for MVP. -- Forgejo access is performed by a constrained service account with minimum PAT scopes. -- A repo is shareable only if the service account can see it and Embargo has sharing enabled for it. +## Features +### Two Link Types + +| Type | Verification | Use Case | +|------|--------------|----------| +| **Private** | Email verification code required | Share with specific individuals | +| **Public** | No verification, time-expired | Job applications, portfolios | + +### Security + +- Links pinned to immutable `commit_sha` — not moving branches +- Email verification prevents unauthorized access +- Time-expiration and revocation give owners control +- No public repo listing — anti-scraping by design +- Admin access protected upstream (Tailscale, VPN, Cloudflare Access) + +### Authentication + +| Purpose | Method | +|---------|--------| +| User Identity | Forgejo Sessions | +| Data Access | Service Account PAT | + +### Tech Stack + +- **Backend**: Python + FastAPI +- **Frontend**: Jinja2 templates + HTMX (no build step) +- **Database**: PostgreSQL + +HTMX provides interactivity (filtering, form submissions) with minimal JavaScript. Jinja2 handles server-side rendering for fast page loads. + +## Quick Start + +### 1. Run `myst init` + +```bash +myst init +``` + +Prompts for: + +- Public viewer URL (`share.example.com`) +- Private admin URL (`admin.example.com`) +- Forgejo base URL (`git.example.com`) +- PostgreSQL connection string +- Forgejo service account username and PAT +- Grant token secret + +### 2. Configure Service Account + +Create a dedicated Forgejo user (e.g., `myst-bot`) with a PAT: + +- `read:user` +- `read:repository` + +### 3. Deploy + +Deploy via Dokploy, Coolify, or your preferred platform. + +## Example Configuration + +```env +MYST_PUBLIC_URL="https://share.example.com" +MYST_ADMIN_URL="https://admin.example.com" +DATABASE_URL="postgresql://myst:secret@localhost:5432/myst" +FORGEJO_BASE_URL="https://git.example.com" +FORGEJO_SERVICE_ACCOUNT_USERNAME="myst-bot" +FORGEJO_PAT="fgp_xxxxxxxxxxxxxxxxxxxx" +GRANT_TOKEN_SECRET="your-secret-here" +``` + +## Phase 1 Progress + +- [x] Project renamed from Embargo to Myst +- [x] CLI scaffold (`myst init`) +- [x] Config generation (`.env`, `myst.config.json`) +- [ ] Service foundation (FastAPI + PostgreSQL) +- [ ] `myst forgejo bootstrap` — verify Forgejo connectivity +- [ ] `myst doctor` — validate configuration +- [ ] `myst config print` — show current config +- [ ] Forgejo session authentication +- [ ] Repo ownership verification +- [ ] Private link creation (email-verified) +- [ ] Public link creation (time-expired) +- [ ] Web-based code viewer (Jinja2 + HTMX) +- [ ] User dashboard +- [ ] SMTP configuration UI +- [ ] Access logs + +## Resources + +- [Forgejo Docs](https://forgejo.org/docs/) +- [API Usage](https://forgejo.org/docs/latest/user/api-usage/) +- [Token Scopes](https://forgejo.org/docs/latest/user/token-scope/) diff --git a/docs/plans/2026-03-30-phase-1-design.md b/docs/plans/2026-03-30-phase-1-design.md index acf5eeb..742759c 100644 --- a/docs/plans/2026-03-30-phase-1-design.md +++ b/docs/plans/2026-03-30-phase-1-design.md @@ -1,16 +1,40 @@ -# Embargo Phase 1 Design +# Myst Phase 1 Design -This document captures the approved Phase 1 plan for Embargo. +This document captures the approved Phase 1 plan for Myst. ## Scope Phase 1 delivers: -- CLI bootstrap for Forgejo + Embargo deployment -- Embargo service foundation +- CLI setup for attaching Myst to an existing Forgejo deployment +- Myst service foundation - constrained Forgejo service-account integration - immutable snapshot grants pinned to `commit_sha` - owner dashboard and viewer flow +- two link types: private (email-verified) and public (time-expired) +- web-based code viewer (not git clone URLs) + +Phase 1 does not deliver: + +- Forgejo provisioning +- Forgejo database provisioning +- reverse proxy or DNS automation +- platform-specific hosting orchestration beyond optional templates and docs + +## Tech Stack + +- **Backend**: Python 3.11+ with FastAPI +- **Frontend**: Jinja2 templates with HTMX for interactivity +- **Database**: PostgreSQL +- **Async**: Uses `async/await` with asyncpg for database access + +### Why This Stack? + +**FastAPI**: Lightweight async framework, automatic OpenAPI docs, Pydantic for validation. + +**Jinja2 + HTMX**: Server-side rendering with minimal JavaScript. HTMX handles dynamic updates (filtering, form submissions) without a build step or heavy SPA framework. + +**PostgreSQL**: Reliable, well-supported by async libraries (asyncpg, SQLModel). ## Reviewable Branches @@ -24,6 +48,176 @@ Phase 1 delivers: ## Security Baseline - service account access is restricted to approved repos -- Embargo grant checks are enforced on every Forgejo read +- Myst grant checks are enforced on every Forgejo read - grants store `repo_id` and pinned `commit_sha` - viewer requests never choose repo or ref directly +- operators create the Forgejo service account and PAT themselves +- Myst stores only its own app config and metadata in its own database +- admin access is protected upstream with Tailscale, VPN, Cloudflare Access, reverse-proxy auth, or equivalent operator-managed controls +- Myst does not ship built-in admin auth in Phase 1 + +## Authentication Model + +Myst uses two authentication mechanisms: + +**User Identity (Forgejo Sessions)** +- Users log in via Forgejo's session system +- Myst redirects to Forgejo login and validates the session +- Users use their existing Forgejo accounts — no separate Myst login +- All Forgejo users with an account are trusted Myst users + +**Data Access (Service Account PAT)** +- Myst uses an operator-created service account PAT to call the Forgejo API +- PAT is used to verify repo ownership and fetch code for viewers +- PAT is never used for user identity (that's handled by sessions) + +**Why both?** +- Sessions identify who the user is (zach@me.com logged in) +- PAT allows Myst to verify repo ownership (can zach@me.com create view links for this repo?) +- PAT allows Myst to fetch code content for the viewer (get the pinned commit's files) + +## Forgejo Service Account Baseline + +- the Forgejo service account is a normal Forgejo user account dedicated to Myst, such as `myst-bot` +- the operator creates this user in Forgejo and grants repo access explicitly +- the operator creates the PAT for this user through the Forgejo UI or admin CLI +- recommended PAT scopes are `read:user` and `read:repository` +- `read:organization` may also be required when repo visibility depends on org or team APIs +- Myst must not create Forgejo users, PATs, or repo permissions in Phase 1 + +## Link Types + +Myst supports two types of view links: + +### Private Links (Email-Verified) + +- Time expired (configurable duration) +- Revocable by the owner +- Requires email verification code to view +- Tied to a specific email address +- Owner specifies the recipient's email when creating the link +- Myst sends an email with a verification code +- Viewer enters the code to access the snapshot + +### Public Links (Time-Expired) + +- Time expired (configurable duration) +- Revocable by the owner +- No verification required +- Anyone with the link can view +- Suitable for job application portfolio fields +- Owner copies the link and shares it directly + +## Link Creation Rules + +Only repo owners can create view links for their repos. + +- Myst verifies ownership via the Forgejo API using the service account PAT +- Collaborators or read-access users cannot create view links +- Site admins are treated as regular users for this purpose (details to be finalized) + +## CLI Init Baseline + +`myst init` should ask only for: + +- public viewer URL, for example `https://share.example.com` +- private admin URL, for example `https://admin.example.com` or `https://myst-admin.tailnet.ts.net` +- Forgejo base URL, for example `https://git.example.com` +- Myst Postgres URL, for example `postgresql://myst:password@db.example.com:5432/myst` +- Forgejo service account username, for example `myst-bot` +- Forgejo PAT for that service account +- grant token secret, with secure auto-generation as the default + +`myst init` should generate all Myst-owned deployment config needed for Phase 1: + +- `.env` +- `myst.config.json` +- any future Myst-only deployment templates, but never Forgejo provisioning files + +`myst init` should provide inline guidance that: + +- it should be run in the directory where Myst will be deployed, usually on the target VPS or server workspace +- the admin URL must be protected upstream because Myst does not provide admin auth in Phase 1 +- the database URL must point to Myst's own Postgres database, not Forgejo's tables +- the PAT should be created for the dedicated service account user +- the operator can consult Forgejo docs for token creation and scopes if needed +- bare hostnames such as `share.example.com` are accepted and should default to `https://` + +## Deployment Baseline + +- Forgejo is already deployed and operator-managed +- Myst is deployed independently as its own app +- Myst uses its own Postgres database on the same DB server as Forgejo (separate database, not shared tables) +- Myst reaches Forgejo via API using a least-privilege service account token +- the CLI validates configuration and connectivity rather than provisioning infrastructure +- the Myst admin surface must be protected by VPN, Tailscale, Cloudflare Access, or reverse-proxy auth in front of the app + +### Docker and Deployment + +Myst provides a Dockerfile for containerized deployment: + +- Myst includes a `Dockerfile` in its repository for building the container image +- Operators use their preferred platform (Dokploy, Coolify, manual Docker, etc.) to deploy the Myst container +- Myst does not generate Docker Compose files for Forgejo or other infrastructure +- Myst does not provision containers, images, or hosting resources +- Operators manage their own container orchestration (Dokploy handles this for most users) + +For Dokploy users, deploying Myst is similar to deploying any other containerized application: + +1. Build or import the Myst image using Dokploy's Docker Compose support +2. Configure environment variables from `.env` generated by `myst init` +3. Set up the public and admin domains in Dokploy's domain configuration +4. Dokploy handles Traefik routing, TLS, and container restarts + +## Database Model + +Myst uses its own Postgres database on the same DB server as Forgejo: + +- separate databases (not shared tables with Forgejo) +- recommended: same PostgreSQL instance with `forgejo` and `myst` databases +- clean schema separation prevents coupling to Forgejo's database schema +- independent backups and migrations +- operators can run both databases on a small VPS efficiently + +Minimum v1 data model: + +- `grants`: view link metadata (repo, commit_sha, type, expiration, email, revoked) +- `verification_codes`: email verification codes for private links +- `sessions`: user sessions (validated against Forgejo) +- `smtp_config`: SMTP settings (encrypted) + +## Forgejo Bootstrap Baseline + +`myst forgejo bootstrap` should verify, not provision: + +- Forgejo base URL reachability +- service account token validity +- authenticated user matches the configured service account username +- minimum token scopes are sufficient for Myst's read-only API usage +- Myst can resolve repo metadata, pinned commits, and tree/file reads for the repos it is expected to serve +- Myst can verify repo ownership for the operator's user account + +## Viewer Baseline + +The web-based viewer is the public-facing code display surface: + +- displays code with syntax highlighting +- read-only, no raw git access +- validates link token, expiration, and revoked state +- for private links: requires valid email verification code +- for public links: accessible immediately with the link +- never exposes Forgejo admin surfaces or database ports +- pinned to immutable commit_sha (not moving branches) + +## User Dashboard + +The owner dashboard allows repo owners to manage their view links: + +- list of created links (public and private) +- link status (active, expired, revoked) +- view count and access logs (details to be finalized) +- create new link flow +- revoke/delete existing links +- SMTP configuration page + +First-time users see a quick setup page to configure SMTP credentials before accessing the full dashboard. diff --git a/docs/plans/2026-03-31-myst-auth-and-deployment-design.md b/docs/plans/2026-03-31-myst-auth-and-deployment-design.md new file mode 100644 index 0000000..242e377 --- /dev/null +++ b/docs/plans/2026-03-31-myst-auth-and-deployment-design.md @@ -0,0 +1,366 @@ +# Myst Auth and Deployment Design + +This document captures current decisions about deployment topology, authentication, and CLI scope for Phase 1. + +## Goals + +- Keep Myst self-hosted and private by default. +- Let operators attach Myst to an existing Forgejo deployment. +- Expose only the read-only share-link viewer and Myst UI routes the operator chooses to publish. +- Make the CLI responsible for Myst-specific configuration, validation, and Forgejo integration checks. +- Keep Phase 1 scope minimal and shippable. +- Prevent code theft and scraping by controlling who can view shared links and how. + +## Non-Goals + +- Built-in admin authentication in Myst for Phase 1. +- Provisioning Forgejo for operators. +- Provisioning PostgreSQL for Forgejo or Myst. +- Owning reverse proxy, DNS, TLS, backup, or firewall setup. +- Replacing Dokploy, Coolify, or any other hosting platform. +- Adding a SaaS control plane. +- Requiring end users to join Tailscale in order to view shared code. +- Designing a browser-based admin UX before the backend and integration flow exist. + +## Core Architecture + +Myst is a companion service deployed beside an existing Forgejo instance. It owns its own UI/API app and its own database, and it integrates with Forgejo only through the Forgejo API. + +```ascii + Public Internet + | + v + DNS: share.example.com (viewer) + DNS: admin.example.com (dashboard, protected) + | + v + Reverse Proxy / Platform Router + (Dokploy, Caddy, Nginx, etc.) + | + v + Myst App (Viewer + Dashboard) + | + ------------------- + | | + v v + Myst Database Forgejo API + (Postgres) (existing instance) +``` + +Deployment boundaries: + +- Forgejo is already running and is managed by the operator. +- Myst is deployed independently from Forgejo. +- Myst never reads or writes Forgejo's database directly. +- Myst uses a service account PAT to call the Forgejo API. +- Myst uses Forgejo sessions to identify logged-in users. +- Myst admin access is protected upstream rather than by an in-app login system in Phase 1. +- VPN, Tailscale, Cloudflare Access, or reverse-proxy auth is required for the admin surface in Phase 1. + +## Authentication + +Myst uses two authentication mechanisms for different purposes: + +### User Identity: Forgejo Sessions + +Users log in via Forgejo's existing session system: + +1. User visits Myst dashboard +2. Myst redirects to Forgejo login page +3. User authenticates with Forgejo credentials +4. Forgejo sets a session cookie +5. Myst validates the session via Forgejo API +6. Myst knows the user's identity without maintaining its own user database + +**Benefits:** +- No separate Myst login — users use their existing Forgejo account +- All Forgejo users with an account are trusted Myst users +- No separate user management in Myst + +### Data Access: Service Account PAT + +Myst uses an operator-created PAT to call the Forgejo API: + +1. Verify repo ownership when owner creates a view link +2. Fetch code content when serving view links to viewers + +**Why both mechanisms?** +- Sessions identify the user (zach@me.com is logged in) +- PAT authorizes Myst's server-to-server API calls (fetch repo data, verify ownership) +- PAT is not used for user identity — that's the session's job + +## Link Types + +### Private Links (Email-Verified) + +Intended for sharing with specific individuals via email. + +**Creation:** +- Owner selects repo and commit_sha +- Owner enters recipient's email address +- Owner sets expiration time +- Myst generates a unique link token + +**Viewer flow:** +1. Owner shares the link with the recipient (via email or copy) +2. Recipient clicks the link +3. Recipient enters email verification code from Myst's email +4. On successful verification, recipient can view the code snapshot + +**Protections:** +- Link is useless to anyone without the verification code +- Cannot scrape or index the content +- Owner can revoke access at any time + +### Public Links (Time-Expired) + +Intended for job applications, portfolios, and public sharing. + +**Creation:** +- Owner selects repo and commit_sha +- Owner sets expiration time (e.g., 7 days) +- No email required + +**Viewer flow:** +1. Owner copies the link +2. Pastes into a job application field or shares publicly +3. Anyone with the link can view immediately (no verification) + +**Protections:** +- Link expires automatically +- Owner can revoke at any time +- No public listing of repos or profiles + +## Anti-Scraping Protections + +Myst is designed to prevent code theft and scraping: + +- No public listing of repositories +- No profile exposure +- Private links require email verification +- Public links are time-limited and revocable +- Links are tied to immutable commit snapshots +- Cannot dig through repo list or browse content without a valid link + +## Deployment + +Myst should be deployable anywhere an operator can run a normal web app and Postgres database. + +Recommended v1 deployment environments: + +1. Same VPS or host as Forgejo (via Dokploy) +2. Same Dokploy project or private network as Forgejo +3. Separate host on the same trusted network as Forgejo + +### Docker and Deployment + +Myst provides a Dockerfile for containerized deployment: + +- Myst includes a `Dockerfile` in its repository for building the container image +- Operators use their preferred platform (Dokploy, Coolify, manual Docker, etc.) to deploy the Myst container +- Myst does not generate Docker Compose files for Forgejo or other infrastructure +- Myst does not provision containers, images, or hosting resources +- Operators manage their own container orchestration + +For Dokploy users, deploying Myst is similar to deploying any other containerized application: + +1. Build or import the Myst image using Dokploy's Docker Compose support +2. Configure environment variables from `.env` generated by `myst init` +3. Set up the public and admin domains in Dokploy's domain configuration +4. Dokploy handles Traefik routing, TLS, and container restarts + +### Database Configuration + +Myst uses its own Postgres database on the same DB server as Forgejo: + +- Separate databases (e.g., `forgejo` and `myst`) +- NOT shared tables or schemas +- Operators can run both on a small VPS efficiently +- Independent backups and migrations + +Admin access security is the operator's responsibility (VPN, firewall rules, SSO, host access, etc). Myst does not enforce or scaffold this in Phase 1. + +Recommended admin protection modes: + +1. Tailscale or another private network +2. Cloudflare Access or similar identity-aware proxy +3. Reverse-proxy basic auth or IP allowlisting when the operator accepts the trade-offs + +Recommended hostname split: + +- `https://share.example.com` for the public viewer +- `https://admin.example.com` for the private admin UI + +## CLI Responsibilities + +The CLI is the primary way to configure and validate Myst against an existing Forgejo deployment. + +### `init` + +`init` should: + +- prompt for the public viewer URL, for example `https://share.example.com` +- prompt for the private admin URL, for example `https://admin.example.com` +- prompt for the Forgejo base URL, for example `https://git.example.com` +- collect the Forgejo service account username, for example `myst-bot` +- collect a user-created Forgejo PAT for that service account +- collect Myst Postgres connection details, for example `postgresql://myst:password@db.example.com:5432/myst` +- generate a grant token secret by default unless the operator pastes one +- generate all Myst-owned config files needed for deployment in Phase 1 +- write `.env` +- write `myst.config.json` +- print manual follow-up actions for admin URL protection, app deployment, and Forgejo validation +- show links to Forgejo docs for API usage, token scopes, repo permissions, and the admin CLI +- remind operators to run `myst init` in the directory where Myst will be deployed, usually on the target VPS or server workspace + +**Setup is manual** — operators enter all values directly. No auto-detection of Forgejo configuration. + +Suggested `init` prompt guidance: + +- Public viewer URL: "What public URL should viewers use for shared links?" Example: `https://share.example.com` +- Private admin URL: "What private URL should operators use for the Myst admin UI?" Example: `https://admin.example.com` +- Forgejo base URL: "What is the base URL of your existing Forgejo instance?" Example: `https://git.example.com` +- Myst database URL: "What Postgres connection string should Myst use?" Example: `postgresql://myst:password@db.example.com:5432/myst` +- Forgejo service account username: "What Forgejo username should Myst use for API access?" Example: `myst-bot` +- Forgejo PAT: "Paste the Forgejo personal access token for that service account" Example: `fgp_...` +- Grant token secret: "Paste a grant token secret, or press enter to generate one automatically" + +Suggested `init` prompt notes: + +- "Protect the admin URL upstream with Tailscale, VPN, Cloudflare Access, or reverse-proxy auth. Myst does not provide admin auth in Phase 1." +- "Use a dedicated Forgejo user for Myst and create the PAT as that user." +- "Recommended PAT scopes: `read:user`, `read:repository`; add `read:organization` only if your org or team setup requires it." +- "Do not point Myst at Forgejo's database tables." +- "Bare hostnames such as `share.example.com` are accepted and default to `https://`. Enter the full URL only if you need `http://` or a custom path." +- "Run `myst init` in the directory where you plan to deploy Myst so the generated files are already in place on the target host." + +Forgejo docs to link directly from CLI output: + +- `https://forgejo.org/docs/latest/user/api-usage/` +- `https://forgejo.org/docs/latest/user/token-scope/` +- `https://forgejo.org/docs/latest/user/repo-permissions/` +- `https://forgejo.org/docs/latest/admin/command-line/` + +### `forgejo bootstrap` + +`forgejo bootstrap` should: + +- validate Forgejo connectivity +- validate the provided service account token +- verify minimum required scopes and permissions +- confirm Myst can resolve the repos it needs +- confirm the authenticated user matches the configured service account username +- confirm Myst can resolve a branch or ref to a pinned `commit_sha` +- confirm Myst can read tree and file data for a pinned commit +- verify Myst can confirm repo ownership for the operator's user account +- never create Forgejo users, PATs, or repo permissions + +### `doctor` + +`doctor` should: + +- validate the local Myst config +- validate database connectivity +- validate Forgejo API reachability +- validate token/scopes/permissions +- validate the public base URL and key route assumptions + +### `config print` + +`config print` should: + +- show resolved Myst configuration +- redact secrets by default +- help operators debug misconfiguration without touching infrastructure + +### CLI UX + +- Use spinners for long-running steps. +- Use prompts for interactive configuration. +- Keep output terse and actionable. +- Present required manual follow-up actions clearly after initialization. + +## Configuration Outputs + +Expected outputs from `init`: + +- `.env` +- `myst.config.json` +- optional platform templates for Myst only (e.g., Docker Compose for Dokploy) + +## Data Model + +Myst stores its metadata in its own Postgres database (separate from Forgejo's database). + +Minimum v1 data model: + +- `grants`: view link metadata (repo_id, commit_sha, type, expiration, email, revoked, created_by) +- `verification_codes`: email verification codes for private links (code, email, grant_id, expires_at) +- `sessions`: user sessions (session_token, forgejo_user_id, created_at, expires_at) +- `smtp_config`: SMTP settings (encrypted, configured via dashboard) + +Key constraints: + +- grants are tied to immutable commit snapshots +- viewer endpoints never accept arbitrary repo/ref selection +- link tokens are unique and unguessable +- verification codes expire quickly (e.g., 15 minutes) + +## User Dashboard + +The owner dashboard provides: + +- List of created links (public and private) +- Link status (active, expired, revoked) +- View count and access logs (details to be finalized) +- Create new link flow: + - Select repo (fetched via service account PAT, filtered to owned repos) + - Select commit (branch/tag to SHA resolution) + - Choose link type (public or private) + - Set expiration + - For private: enter recipient email +- Revoke/delete existing links +- SMTP configuration page + +**First-time setup:** +- After logging in via Forgejo, users see a quick setup page to configure SMTP credentials +- Once configured, full dashboard access is granted + +## Viewer Rules + +The viewer is the only public application surface that emits code content. + +- Share links are tied to immutable snapshots (commit_sha) +- Viewer requests must validate: + - link token + - expiration + - revoked state + - snapshot binding + - for private links: valid email verification code +- Public access must be read-only +- Public access must not expose database ports or Forgejo admin surfaces + +## Repo Implementation Impact + +Active surfaces: + +- CLI entrypoint: `packages/cli/src/index.ts` +- Command stubs: `packages/cli/src/commands/*` +- Config generators: `packages/cli/src/generators/*` +- Shared constants/types: `packages/shared/src/*` +- Service entrypoint: `packages/service/src/index.ts` +- Viewer routes: `packages/service/src/viewer/*` +- Dashboard routes: `packages/service/src/dashboard/*` + +Next implementation work: + +1. Narrow `init` to Myst-only config generation for an existing Forgejo deployment +2. Add config types for Forgejo connection details, Myst base URL, database, and SMTP settings +3. Implement `forgejo bootstrap` and `doctor` +4. Add Postgres-backed service foundation using the data model +5. Wire the service to load generated config and validate required env at startup +6. Implement Forgejo session authentication for the dashboard +7. Build the link creation flow with repo ownership verification +8. Implement the web-based code viewer +9. Build the user dashboard with access logs +10. Implement email sending for private link verification codes diff --git a/package.json b/package.json index 5548997..440aabe 100644 --- a/package.json +++ b/package.json @@ -1,8 +1,8 @@ { - "name": "embargo", + "name": "myst", "private": true, "version": "0.1.0", - "packageManager": "pnpm@10.11.0", + "packageManager": "pnpm@10.33.0", "devDependencies": { "@types/node": "^24.0.0", "typescript": "^5.8.3" diff --git a/packages/cli/examples/myst.config.example.json b/packages/cli/examples/myst.config.example.json new file mode 100644 index 0000000..f0c680e --- /dev/null +++ b/packages/cli/examples/myst.config.example.json @@ -0,0 +1,6 @@ +{ + "adminUrl": "https://admin.example.com", + "forgejoBaseUrl": "https://git.example.com", + "forgejoServiceAccountUsername": "myst-bot", + "publicUrl": "https://share.example.com" +} diff --git a/packages/cli/package.json b/packages/cli/package.json index 09d80e7..5eff730 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -1,10 +1,19 @@ { - "name": "@embargo/cli", + "name": "@myst/cli", + "description": "A CLI tool that helps you wire Myst into an existing Forgejo instance with expiring grant access controls for repos you choose to share.", "version": "0.1.0", + "author": { + "name": "Zach Tatman", + "email": "zach.c.tatman@gmail.com" + }, + "bugs": { + "url": "https://github.com/ZTatman/myst/issues" + }, + "homepage": "https://github.com/ZTatman/myst#readme", "private": true, "type": "module", "bin": { - "embargo": "./dist/index.js" + "myst": "./dist/index.js" }, "scripts": { "build": "tsc -p tsconfig.json", @@ -12,6 +21,8 @@ "clean": "rm -rf dist" }, "dependencies": { - "@embargo/shared": "workspace:*" + "@clack/prompts": "^1.2.0", + "@myst/shared": "workspace:*", + "yoctocolors": "^2.1.2" } } diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts new file mode 100644 index 0000000..56aa97c --- /dev/null +++ b/packages/cli/src/commands/init.ts @@ -0,0 +1,273 @@ +import fs from "node:fs/promises"; +import path from "node:path"; + +import * as c from "yoctocolors"; +import * as p from "@clack/prompts"; + +import { generateSecret } from "../utils/crypto.js"; +import { generateEnvFile } from "../utils/env.js"; + +/** + * Normalizes a URL to use https:// if no protocol is specified. + * @param value - The URL to normalize. + * @returns {string} The normalized URL. + */ +function normalizeHttpUrl(value: string): string { + const trimmed = value.trim(); + const hasProtocol = /^[a-z]+:\/\//i.test(trimmed); + const isNotHttp = !/^https?:\/\//i.test(trimmed); + + if (hasProtocol && isNotHttp) { + throw new Error("URLs must use http:// or https://"); + } + + return hasProtocol ? trimmed : `https://${trimmed}`; +} + +/** + * Validates a URL or host, returning the normalized URL or undefined if valid. + * @param value - The URL or host to validate. + * @param label - The label to use in error messages. + * @returns {string | undefined} The normalized URL or undefined if valid. + */ +function validateHttpUrlOrHost( + value: string | undefined, + label: string, +): string | undefined { + if (!value?.trim()) return `${label} is required`; + + try { + const normalized = normalizeHttpUrl(value); + const url = new URL(normalized); + + if (!url.hostname) { + return `${label} must include a domain or hostname`; + } + } catch { + return `${label} must be a valid URL or hostname`; + } + + return undefined; +} + +/** + * Validates a Postgres URL, returning undefined if valid. + * @param value - The Postgres URL to validate. + * @returns {string | undefined} The error message or undefined if valid. + */ +function validatePostgresUrl(value: string | undefined): string | undefined { + if (!value?.trim()) { + return "Database URL is required"; + } + + try { + const url = new URL(value); + if (url.protocol !== "postgres:" && url.protocol !== "postgresql:") { + return "Database URL must start with postgres:// or postgresql://"; + } + } catch { + return "Database URL must be a valid Postgres connection string"; + } + + return undefined; +} + +/** + * Writes a private file to disk, throwing an error if the file already exists. + * @param filePath - The path to the file to write. + * @param content - The content to write to the file. + * @returns {Promise} A promise that resolves when the file is written. + */ +async function writePrivateFile( + filePath: string, + content: string, +): Promise { + try { + await fs.writeFile(filePath, content, { + encoding: "utf8", + flag: "wx", + mode: 0o600, + }); + } catch (e) { + const error = e as NodeJS.ErrnoException; + if (error.code === "EEXIST") { + throw new Error( + `Error: the file '${c.underline(path.basename(filePath))}' already exists in directory ${c.underline(path.dirname(filePath))}`, + ); + } + throw error; // throw any other errors back if not EEXIST code + } +} + +/** + * Preflight check for write targets, throwing an error if any target file already exists. + * @param filePaths - The paths of the files to write. + * @returns {Promise} A promise that resolves when all targets are checked. + */ +async function preflightWriteTargets(filePaths: string[]): Promise { + for (const filePath of filePaths) { + try { + await fs.access(filePath); + throw new Error( + `Error: the file '${c.underline(path.basename(filePath))}' already exists in directory ${c.underline(path.dirname(filePath))}`, + ); + } catch (error) { + const err = error as NodeJS.ErrnoException; + if (err.code !== "ENOENT") { + throw error; + } + } + } +} + +/** + * Initializes a Myst project by creating the necessary configuration files. + * @param _args - The command arguments. + * @returns {Promise} A promise that resolves when the initialization is complete. + */ +export async function initCommand(_args: string[]): Promise { + p.intro(c.bold("myst init")); + p.note( + [ + `${c.bold("Run this in the directory where you plan to deploy Myst.")}`, + "", + c.bold("Forgejo docs:"), + "", + `- API usage: ${c.cyan("https://forgejo.org/docs/latest/user/api-usage/")}`, + `- Token scopes: ${c.cyan("https://forgejo.org/docs/latest/user/token-scope/")}`, + `- Repo permissions: ${c.cyan("https://forgejo.org/docs/latest/user/repo-permissions/")}`, + `- Admin CLI: ${c.cyan("https://forgejo.org/docs/latest/admin/command-line/")}`, + "", + `1. Create a dedicated Forgejo user such as ${c.bold("myst-bot")} and generate its PAT in Forgejo ${c.bold("Settings -> Applications")}.`, + `2. Generate a PAT with the appropriate scopes for your Forgejo setup.`, + "", + `Recommended PAT scopes:\n - ${c.bold("read:user")}\n - ${c.bold("read:repository")}\n - ${c.bold("read:organization")} (optional)`, + ].join("\n"), + c.yellow("Before you start:"), + ); + + const answers = await p.group( + { + publicUrl: () => + p.text({ + message: "Domain name or URL for the public viewer", + placeholder: "share.example.com", + validate: (v) => validateHttpUrlOrHost(v, "Public viewer URL"), + }), + adminUrl: () => + p.text({ + message: "Domain name or URL for the Myst admin UI", + placeholder: "admin.example.com", + validate: (v) => validateHttpUrlOrHost(v, "Private admin URL"), + }), + forgejoBaseUrl: () => + p.text({ + message: "Domain name or URL for your existing Forgejo instance?", + placeholder: "git.example.com", + validate: (v) => validateHttpUrlOrHost(v, "Forgejo base URL"), + }), + databaseUrl: () => + p.text({ + message: "What Postgres connection string should Myst use?", + placeholder: "postgresql://myst:password@db.example.com:5432/myst", + validate: validatePostgresUrl, + }), + forgejoServiceAccountUsername: () => + p.text({ + message: + "What Forgejo service account username should Myst use for API access?", + placeholder: "myst-bot", + validate: (v) => + v?.trim() + ? undefined + : "Forgejo service account username is required", + }), + forgejoPat: () => + p.password({ + message: + "Paste the Forgejo personal access token for that service account", + validate: (v) => (v?.trim() ? undefined : "Forgejo PAT is required"), + }), + grantTokenSecret: () => + p.password({ + message: + "Paste a grant token secret, or press enter to generate one automatically", + validate: () => undefined, + }), + }, + { + onCancel: () => { + p.cancel("\nOperation cancelled."); + process.exit(0); + }, + }, + ); + + const s = p.spinner(); + s.start("Generating config files"); + const publicUrl = normalizeHttpUrl(answers.publicUrl); + const adminUrl = normalizeHttpUrl(answers.adminUrl); + const forgejoBaseUrl = normalizeHttpUrl(answers.forgejoBaseUrl); + const grantTokenSecret = answers.grantTokenSecret?.trim() || generateSecret(); + const envContent = generateEnvFile({ + adminUrl, + databaseUrl: answers.databaseUrl, + forgejoBaseUrl, + forgejoPat: answers.forgejoPat, + forgejoServiceAccountUsername: answers.forgejoServiceAccountUsername, + grantTokenSecret, + publicUrl, + }); + const configContent = JSON.stringify( + { + adminUrl, + forgejoBaseUrl, + forgejoServiceAccountUsername: answers.forgejoServiceAccountUsername, + publicUrl, + }, + null, + 2, + ); + + const outDir = process.cwd(); + const envPath = path.join(outDir, ".env"); + const configPath = path.join(outDir, "myst.config.json"); + + try { + await preflightWriteTargets([envPath, configPath]); + await writePrivateFile(envPath, envContent); + await writePrivateFile(configPath, configContent); + } catch (e) { + s.stop(); + p.cancel( + e instanceof Error + ? e.message + : `Failed to write config files to ${outDir}`, + ); + return; + } + + s.stop("Config files generated"); + + p.note( + [c.green(".env"), c.green("myst.config.json")].join("\n"), + c.yellow("Files written to current directory"), + ); + + p.note( + [ + `${c.bold("Public viewer URL:")} ${c.cyan(publicUrl)}`, + `${c.bold("Private admin URL:")} ${c.cyan(adminUrl)}`, + `${c.bold("Forgejo base URL:")} ${c.cyan(forgejoBaseUrl)}`, + `${c.bold("Forgejo service account:")} ${c.green(answers.forgejoServiceAccountUsername)}`, + "", + `${c.bold("Admin URL reminder:")} protect this upstream with Tailscale, VPN, Cloudflare Access, or reverse-proxy auth.`, + ].join("\n"), + c.yellow("Next steps"), + ); + p.outro( + c.green( + "Done. Deploy Myst, then run forgejo bootstrap and doctor commands to verify setup.", + ), + ); +} diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index f7c256f..88a6a00 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -1,14 +1,113 @@ -import { EMBARGO_DEFAULT_FORGEJO_PORT, EMBARGO_DEFAULT_SERVICE_PORT } from "@embargo/shared"; +#!/usr/bin/env node +import * as c from "yoctocolors"; +import pkg from "../package.json" with { type: "json" }; -function main(): void { - process.stdout.write( - [ - "Embargo CLI bootstrap", - `Forgejo default port: ${EMBARGO_DEFAULT_FORGEJO_PORT}`, - `Service default port: ${EMBARGO_DEFAULT_SERVICE_PORT}` - ].join("\n") + "\n" - ); +import { initCommand } from "./commands/init.js"; + +const PADDING = 4; +const INDENT = " "; +const USAGE = "myst [options/flags] [arguments]"; + +type CommandName = "init"; + +interface Option { + name: CommandName | string | string[]; + desc: string; } -main(); +interface CLIConfig { + commands: Option[]; + globalFlags: Option[]; +} +const config: CLIConfig = { + commands: [ + { name: "init", desc: "Bootstrap Myst and generate config/env files" }, + ], + globalFlags: [ + { name: ["-h", "--help"], desc: "Display help information" }, + { name: ["-v", "--version"], desc: "Display package version number" }, + ], +}; + +const splitCamelCaseWords = (text: string): string => + text.replace(/([a-z])([A-Z])/g, "$1 $2"); + +const toTitleCase = (text: string): string => + text.replace(/\b\w/g, (c) => c.toUpperCase()); + +function formatSection( + header: string, + options: Option[], + colWidth: number, +): string { + const rows = options.map((o) => { + const name = Array.isArray(o.name) ? o.name.join(", ") : o.name; + return `${INDENT}${name.padEnd(colWidth)}${o.desc}`; + }); + const title = toTitleCase(splitCamelCaseWords(header)); + return [c.bold(title + ":"), ...rows].join("\n"); +} + +const allOptions = Object.values(config).flat(); +const COL_WIDTH = + Math.max( + ...allOptions.map((o) => + Array.isArray(o.name) ? o.name.join(", ").length : o.name.length, + ), + ) + PADDING; + +const HELP_TEXT = ` +${c.yellow(pkg.name)} + + +${INDENT}${USAGE} + + +${Object.entries(config) + .map(([header, options]) => + formatSection(header, options as Option[], COL_WIDTH), + ) + .join("\n\n")} +`; + +async function main(): Promise { + const [command, ...options] = process.argv.slice(2); + const flags = options.filter((o) => o.startsWith("-")); + + if (flags.includes("-v") || flags.includes("--version")) { + process.stdout.write(`${pkg.version}\n`); + process.exitCode = 0; + return; + } + + if (!command || flags.includes("-h") || flags.includes("--help")) { + process.stdout.write(`${HELP_TEXT}`); + process.exitCode = 0; + return; + } + + const knownCommands: Record Promise> = + { + init: initCommand, + }; + + const isKnownCommand = (cmd: string): cmd is CommandName => + cmd in knownCommands; + + if (!isKnownCommand(command)) { + process.stderr.write(`Unknown command: ${command}\n`); + process.exitCode = 1; + return; + } + + try { + await knownCommands[command](options); + } catch (error) { + const message = error instanceof Error ? error.message : "Unknown error"; + process.stderr.write(`${c.red("Error:")} ${message}\n`); + process.exitCode = 1; + } +} + +await main(); diff --git a/packages/cli/src/utils/crypto.ts b/packages/cli/src/utils/crypto.ts new file mode 100644 index 0000000..43dce1c --- /dev/null +++ b/packages/cli/src/utils/crypto.ts @@ -0,0 +1,9 @@ +import { randomBytes } from "node:crypto"; + +export function generateSecret(bytes = 32): string { + return randomBytes(bytes).toString("hex"); +} + +export function generatePassword(bytes = 16): string { + return randomBytes(bytes).toString("base64url"); +} diff --git a/packages/cli/src/utils/env.ts b/packages/cli/src/utils/env.ts new file mode 100644 index 0000000..38712f9 --- /dev/null +++ b/packages/cli/src/utils/env.ts @@ -0,0 +1,41 @@ +interface EnvOptions { + adminUrl: string; + databaseUrl: string; + forgejoBaseUrl: string; + forgejoPat: string; + forgejoServiceAccountUsername: string; + grantTokenSecret: string; + publicUrl: string; +} + +function escapeEnvValue(value: string): string { + return `"${value + .replace(/\\/g, "\\\\") // Escape backslashes + .replace(/"/g, '\\"') // Escape double quotes + .replace(/\r?\n/g, "\\n")}"`; // Escape newlines +} + +/** + * Generates an .env file for Myst with the given options. + * @param options - The options for generating the .env file. + * @returns {string} The .env file formatted as a string. + */ +export function generateEnvFile(options: EnvOptions): string { + return `# Public URL used for share links. +MYST_PUBLIC_URL=${escapeEnvValue(options.publicUrl)} + +# Private operator URL for the admin UI. Protect this upstream. +MYST_ADMIN_URL=${escapeEnvValue(options.adminUrl)} + +# Myst's own Postgres database. +DATABASE_URL=${escapeEnvValue(options.databaseUrl)} + +# Existing Forgejo instance and operator-created service account token. +FORGEJO_BASE_URL=${escapeEnvValue(options.forgejoBaseUrl)} +FORGEJO_SERVICE_ACCOUNT_USERNAME=${escapeEnvValue(options.forgejoServiceAccountUsername)} +FORGEJO_PAT=${escapeEnvValue(options.forgejoPat)} + +# Secret used to sign or derive grant tokens. +GRANT_TOKEN_SECRET=${escapeEnvValue(options.grantTokenSecret)} +`; +} diff --git a/packages/service/package.json b/packages/service/package.json index a846861..4479140 100644 --- a/packages/service/package.json +++ b/packages/service/package.json @@ -1,5 +1,5 @@ { - "name": "@embargo/service", + "name": "@myst/service", "version": "0.1.0", "private": true, "type": "module", @@ -9,6 +9,6 @@ "clean": "rm -rf dist" }, "dependencies": { - "@embargo/shared": "workspace:*" + "@myst/shared": "workspace:*" } } diff --git a/packages/service/src/index.ts b/packages/service/src/index.ts index 721b79a..71d14e2 100644 --- a/packages/service/src/index.ts +++ b/packages/service/src/index.ts @@ -1,3 +1,3 @@ -import { EMBARGO_DEFAULT_SERVICE_PORT } from "@embargo/shared"; +import { MYST_DEFAULT_SERVICE_PORT } from "@myst/shared"; -process.stdout.write(`Embargo service bootstrap on port ${EMBARGO_DEFAULT_SERVICE_PORT}\n`); +process.stdout.write(`Myst service bootstrap on port ${MYST_DEFAULT_SERVICE_PORT}\n`); diff --git a/packages/shared/package.json b/packages/shared/package.json index d06a67b..ecf9864 100644 --- a/packages/shared/package.json +++ b/packages/shared/package.json @@ -1,5 +1,5 @@ { - "name": "@embargo/shared", + "name": "@myst/shared", "version": "0.1.0", "private": true, "type": "module", diff --git a/packages/shared/src/constants.ts b/packages/shared/src/constants.ts index f791b0b..93592ca 100644 --- a/packages/shared/src/constants.ts +++ b/packages/shared/src/constants.ts @@ -1,8 +1,7 @@ -export const EMBARGO_DEFAULT_FORGEJO_PORT = 3000; -export const EMBARGO_DEFAULT_SERVICE_PORT = 4000; +export const MYST_DEFAULT_FORGEJO_PORT = 3000; +export const MYST_DEFAULT_SERVICE_PORT = 4000; -export const EMBARGO_VISIBILITY_MODES = { +export const MYST_VISIBILITY_MODES = { TEAM: "team", ALLOWLIST: "allowlist" } as const; - diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9084cf5..9c9ab0e 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -17,13 +17,19 @@ importers: packages/cli: dependencies: - '@embargo/shared': + '@clack/prompts': + specifier: ^1.2.0 + version: 1.2.0 + '@myst/shared': specifier: workspace:* version: link:../shared + yoctocolors: + specifier: ^2.1.2 + version: 2.1.2 packages/service: dependencies: - '@embargo/shared': + '@myst/shared': specifier: workspace:* version: link:../shared @@ -31,9 +37,27 @@ importers: packages: + '@clack/core@1.2.0': + resolution: {integrity: sha512-qfxof/3T3t9DPU/Rj3OmcFyZInceqj/NVtO9rwIuJqCUgh32gwPjpFQQp/ben07qKlhpwq7GzfWpST4qdJ5Drg==} + + '@clack/prompts@1.2.0': + resolution: {integrity: sha512-4jmztR9fMqPMjz6H/UZXj0zEmE43ha1euENwkckKKel4XpSfokExPo5AiVStdHSAlHekz4d0CA/r45Ok1E4D3w==} + '@types/node@24.12.0': resolution: {integrity: sha512-GYDxsZi3ChgmckRT9HPU0WEhKLP08ev/Yfcq2AstjrDASOYCSXeyjDsHg4v5t4jOj7cyDX3vmprafKlWIG9MXQ==} + fast-string-truncated-width@1.2.1: + resolution: {integrity: sha512-Q9acT/+Uu3GwGj+5w/zsGuQjh9O1TyywhIwAxHudtWrgF09nHOPrvTLhQevPbttcxjr/SNN7mJmfOw/B1bXgow==} + + fast-string-width@1.1.0: + resolution: {integrity: sha512-O3fwIVIH5gKB38QNbdg+3760ZmGz0SZMgvwJbA1b2TGXceKE6A2cOlfogh1iw8lr049zPyd7YADHy+B7U4W9bQ==} + + fast-wrap-ansi@0.1.6: + resolution: {integrity: sha512-HlUwET7a5gqjURj70D5jl7aC3Zmy4weA1SHUfM0JFI0Ptq987NH2TwbBFLoERhfwk+E+eaq4EK3jXoT+R3yp3w==} + + sisteransi@1.0.5: + resolution: {integrity: sha512-bLGGlR1QxBcynn2d5YmDX4MGjlZvy2MRBDRNHLJ8VI6l6+9FUiyTFNJ0IveOSP0bcXgVDPRcfGqA0pjaqUpfVg==} + typescript@5.9.3: resolution: {integrity: sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==} engines: {node: '>=14.17'} @@ -42,12 +66,42 @@ packages: undici-types@7.16.0: resolution: {integrity: sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==} + yoctocolors@2.1.2: + resolution: {integrity: sha512-CzhO+pFNo8ajLM2d2IW/R93ipy99LWjtwblvC1RsoSUMZgyLbYFr221TnSNT7GjGdYui6P459mw9JH/g/zW2ug==} + engines: {node: '>=18'} + snapshots: + '@clack/core@1.2.0': + dependencies: + fast-wrap-ansi: 0.1.6 + sisteransi: 1.0.5 + + '@clack/prompts@1.2.0': + dependencies: + '@clack/core': 1.2.0 + fast-string-width: 1.1.0 + fast-wrap-ansi: 0.1.6 + sisteransi: 1.0.5 + '@types/node@24.12.0': dependencies: undici-types: 7.16.0 + fast-string-truncated-width@1.2.1: {} + + fast-string-width@1.1.0: + dependencies: + fast-string-truncated-width: 1.2.1 + + fast-wrap-ansi@0.1.6: + dependencies: + fast-string-width: 1.1.0 + + sisteransi@1.0.5: {} + typescript@5.9.3: {} undici-types@7.16.0: {} + + yoctocolors@2.1.2: {} diff --git a/tsconfig.base.json b/tsconfig.base.json index d99371a..70e6674 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -6,7 +6,7 @@ "lib": ["ES2022"], "baseUrl": ".", "paths": { - "@embargo/shared": ["packages/shared/src/index.ts"] + "@myst/shared": ["packages/shared/src/index.ts"] }, "strict": true, "noUncheckedIndexedAccess": true,