From eb93333b9fc06d677a8d77c2bce3d57d4abe2703 Mon Sep 17 00:00:00 2001 From: shuki Date: Wed, 4 Mar 2026 02:57:47 +0200 Subject: [PATCH] Update documentation: git repo info, schedules, restore, WHM pages - Add Repository section with SSH/HTTP/Web UI URLs - Rewrite Schedules section: decoupled from remotes, hourly/daily/weekly/monthly/custom types - Add schedule types table with cron patterns - Add mailbox restore to commands and restore workflows - Update commands list with full schedule CRUD and comma-separated remotes - Update WHM plugin docs: restore page, schedule toggles, Runner.pm - Update file layout: schedules.d/, rclone.sh, schedule.conf.example - Add uninstall instructions - Remove empty Production Server heading Co-Authored-By: Claude Opus 4.6 --- CLAUDE.md | 14 ++++- README.md | 179 +++++++++++++++++++++++++++++++++++++----------------- 2 files changed, 135 insertions(+), 58 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index e11294f..53f2c80 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -553,7 +553,19 @@ cd whm/gniza-whm/assets && npm install && npm run build:css ### Deploying changes to a server -Use the install script for fresh installs or upgrades: +Commit and push to git, then run the install script on the target server: ```bash +# From dev machine: push changes +git add -A && git commit -m "description" && git push + +# On target server: install/upgrade curl -sSL http://192.168.100.100:3001/shukivaknin/gniza/raw/branch/main/scripts/install.sh | bash ``` + +### Repository + +| | URL | +|---|-----| +| **Git (SSH)** | `ssh://git@192.168.100.100:2222/shukivaknin/gniza.git` | +| **Git (HTTP)** | `http://192.168.100.100:3001/shukivaknin/gniza.git` | +| **Web UI** | http://192.168.100.100:3001/shukivaknin/gniza | diff --git a/README.md b/README.md index 651d142..e366cae 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,14 @@ cPanel Backup, Restore & Disaster Recovery tool. Uses `pkgacct --nocompress --skiphomedir` for account backups, gzips SQL files individually, and transfers everything (including homedirs) to remote destinations with incremental snapshots. Supports three remote types: **SSH** (rsync with hardlink-based `--link-dest`), **Amazon S3** / S3-compatible (via rclone), and **Google Drive** (via rclone). Supports multiple remote destinations with independent schedules and retention policies. +## Repository + +| | URL | +|---|-----| +| **Git (SSH)** | `ssh://git@192.168.100.100:2222/shukivaknin/gniza.git` | +| **Git (HTTP)** | `http://192.168.100.100:3001/shukivaknin/gniza.git` | +| **Web UI** | http://192.168.100.100:3001/shukivaknin/gniza | + ## Installation ```bash @@ -18,11 +26,20 @@ cd gniza sudo bash scripts/install.sh ``` +To uninstall: + +```bash +sudo bash /usr/local/gniza/scripts/uninstall.sh # from installed copy +# or +sudo bash scripts/uninstall.sh # from repo clone +``` + +The uninstall script removes the CLI, WHM plugin, and symlink. Config (`/etc/gniza/`), logs (`/var/log/gniza/`), and cron entries are left for manual cleanup. + ## Quick Start ```bash - -# Interactive setup (creates config + first remote) +# Interactive setup (creates config + first remote + optional schedule) sudo gniza init # Add additional remote destinations @@ -34,24 +51,30 @@ sudo gniza backup --dry-run # Run backup sudo gniza backup -# Install cron schedules for all remotes -sudo gniza schedule install +# Back up to specific remotes +sudo gniza backup --remote=nas,offsite ``` ## Commands ``` -gniza backup [--account=NAME] [--remote=NAME] [--dry-run] -gniza restore account [--remote=NAME] [--timestamp=TS] [--force] -gniza restore files [--remote=NAME] [--path=subpath] [--timestamp=TS] -gniza restore database [--remote=NAME] [--timestamp=TS] -gniza restore server [--remote=NAME] [--timestamp=TS] +gniza backup [--account=NAME] [--remote=NAME[,NAME2]] [--dry-run] +gniza restore account --remote=NAME [--timestamp=TS] [--force] +gniza restore files --remote=NAME [--path=subpath] [--timestamp=TS] +gniza restore database --remote=NAME [--timestamp=TS] +gniza restore mailbox --remote=NAME [--timestamp=TS] +gniza restore server --remote=NAME [--timestamp=TS] gniza list [--account=NAME] [--remote=NAME] gniza verify [--account=NAME] [--remote=NAME] gniza status gniza remote list gniza remote delete -gniza schedule {install|show|remove} +gniza schedule add +gniza schedule delete +gniza schedule list +gniza schedule install +gniza schedule show +gniza schedule remove gniza init gniza init remote gniza version @@ -63,7 +86,7 @@ gniza help | Option | Description | |--------|-------------| | `--config=PATH` | Alternate config file (default: `/etc/gniza/gniza.conf`) | -| `--remote=NAME` | Target a specific remote from `/etc/gniza/remotes.d/` | +| `--remote=NAME[,NAME2]` | Target specific remote(s) from `/etc/gniza/remotes.d/` (comma-separated) | | `--debug` | Enable debug logging | ## Configuration @@ -100,7 +123,7 @@ See `etc/gniza.conf.example` for the full template. ### Remote Destinations -Back up to one or more destinations with independent schedules, retention policies, and bandwidth limits. Supports SSH, Amazon S3 (and S3-compatible services like MinIO, Wasabi, Backblaze B2), and Google Drive. Remote destinations are configured as individual files in `/etc/gniza/remotes.d/`. +Back up to one or more destinations with independent retention policies and bandwidth limits. Supports SSH, Amazon S3 (and S3-compatible services like MinIO, Wasabi, Backblaze B2), and Google Drive. Remote destinations are configured as individual files in `/etc/gniza/remotes.d/`. #### Setup @@ -160,14 +183,15 @@ See `etc/remote.conf.example` for the full template. #### Usage -Without `--remote`, backup/list/verify operate on **all** configured remotes. Restore requires `--remote` to specify the source. +Without `--remote`, backup/list/verify operate on **all** configured remotes. Restore always requires `--remote` to specify the source. ```bash # Back up to all remotes sudo gniza backup -# Back up to a specific remote +# Back up to specific remote(s) sudo gniza backup --remote=nas +sudo gniza backup --remote=nas,offsite # List snapshots on a specific remote sudo gniza list --remote=offsite @@ -176,23 +200,59 @@ sudo gniza list --remote=offsite sudo gniza restore account johndoe --remote=nas ``` -## Schedule Management +### Schedules -Manage cron entries based on each remote's `SCHEDULE` settings. Each remote gets a tagged cron entry for clean install/remove. +Schedules are **decoupled from remotes**. Each schedule lives in `/etc/gniza/schedules.d/.conf` and defines when backups run and which remotes to target. This allows multiple schedules targeting different sets of remotes. + +#### Schedule Config Format ```bash -sudo gniza schedule install # Install cron entries for all remotes -sudo gniza schedule show # Show current gniza cron entries -sudo gniza schedule remove # Remove all gniza cron entries +SCHEDULE="daily" # hourly, daily, weekly, monthly, custom +SCHEDULE_TIME="02:00" # HH:MM (24-hour) +SCHEDULE_DAY="" # hours between backups (1-23) for hourly + # day-of-week (0=Sun..6=Sat) for weekly + # day-of-month (1-28) for monthly +SCHEDULE_CRON="" # Full cron expression for SCHEDULE=custom +REMOTES="" # Comma-separated remote names (empty = all) ``` -Example crontab entries: +#### Managing Schedules + +```bash +# Interactive schedule creation +sudo gniza schedule add nightly + +# List configured schedules +sudo gniza schedule list + +# Delete a schedule +sudo gniza schedule delete nightly + +# Install all schedules to crontab +sudo gniza schedule install + +# Show current gniza cron entries +sudo gniza schedule show + +# Remove all gniza cron entries +sudo gniza schedule remove +``` + +#### Schedule Types + +| Type | SCHEDULE_DAY | Cron Pattern | Example | +|------|-------------|--------------|---------| +| `hourly` | Hours interval (1-23) | ` */ * * *` | Every 2 hours at :15 → `15 */2 * * *` | +| `daily` | — | ` * * *` | Daily at 02:00 → `0 2 * * *` | +| `weekly` | Day-of-week (0-6) | ` * * ` | Sundays at 03:00 → `0 3 * * 0` | +| `monthly` | Day-of-month (1-28) | ` * *` | 1st at 02:00 → `0 2 1 * *` | +| `custom` | — | `SCHEDULE_CRON` (5-field) | `*/30 * * * *` | + +Each schedule gets a tagged cron entry for clean install/remove: ``` -# gniza:nas -0 2 * * * /usr/local/bin/gniza backup --remote=nas >> /var/log/gniza/cron-nas.log 2>&1 -# gniza:offsite -0 3 * * 0 /usr/local/bin/gniza backup --remote=offsite >> /var/log/gniza/cron-offsite.log 2>&1 +# gniza:nightly +0 2 * * * /usr/local/bin/gniza backup --remote=nas,offsite >> /var/log/gniza/cron-nightly.log 2>&1 ``` ## Remote Directory Structure @@ -239,7 +299,7 @@ All rsync operations use `--rsync-path="rsync --fake-super"` to preserve file ow ## Backup Flow 1. Load main config (TEMP_DIR, LOG_DIR, accounts, notifications) -2. Resolve target remotes (`--remote=NAME` or all from `remotes.d/`) +2. Resolve target remotes (`--remote=NAME[,NAME2]` or all from `remotes.d/`) 3. Test connectivity to all targets upfront (SSH or rclone by type) 4. For each account: - `pkgacct` ONCE @@ -258,12 +318,13 @@ If one remote fails for an account, other remotes still receive the backup. ## Restore Workflows -| Workflow | Description | -|----------|-------------| -| **Full account** | Downloads pkgacct data, decompresses SQL, runs `/scripts/restorepkg`, rsyncs homedir, fixes ownership | -| **Selective files** | Rsyncs specific path from remote homedir backup | -| **Single database** | Downloads SQL dump + grants, imports via `mysql` | -| **Full server rebuild** | Restores all accounts found on the remote | +| Workflow | Command | Description | +|----------|---------|-------------| +| **Full account** | `restore account ` | Downloads pkgacct data, decompresses SQL, runs `/scripts/restorepkg`, rsyncs homedir, fixes ownership | +| **Selective files** | `restore files --path=...` | Rsyncs specific path from remote homedir backup | +| **Single database** | `restore database ` | Downloads SQL dump + grants, imports via `mysql` | +| **Mailbox** | `restore mailbox ` | Restores individual email account from `mail///` | +| **Full server** | `restore server` | Restores all accounts found on the remote | All restore commands require `--remote=NAME` to specify the source. @@ -288,6 +349,7 @@ All restore commands require `--remote=NAME` to specify the source. │ ├── config.sh # Config loading and validation │ ├── locking.sh # flock-based concurrency control │ ├── ssh.sh # SSH connectivity, remote_exec, rsync SSH cmd +│ ├── rclone.sh # Rclone transport layer for S3/GDrive │ ├── accounts.sh # cPanel account discovery and filtering │ ├── pkgacct.sh # pkgacct execution, SQL gzip, temp cleanup │ ├── snapshot.sh # Timestamp naming, list/resolve snapshots @@ -295,28 +357,32 @@ All restore commands require `--remote=NAME` to specify the source. │ ├── retention.sh # Prune old snapshots beyond RETENTION_COUNT │ ├── verify.sh # Remote backup integrity checks │ ├── notify.sh # Email notifications -│ ├── restore.sh # Full account, files, database, server restore +│ ├── restore.sh # Full account, files, database, mailbox, server restore │ ├── remotes.sh # Remote discovery and context switching -│ └── schedule.sh # Cron management for multi-remote schedules +│ └── schedule.sh # Cron management for decoupled schedules └── etc/ ├── gniza.conf.example # Main config template - └── remote.conf.example # Remote destination template + ├── remote.conf.example # Remote destination template + └── schedule.conf.example # Schedule template /etc/gniza/ # Runtime configuration ├── gniza.conf # Main config -└── remotes.d/ # Remote destination configs - ├── nas.conf - └── offsite.conf +├── remotes.d/ # Remote destination configs +│ ├── nas.conf +│ └── offsite.conf +└── schedules.d/ # Schedule configs + ├── nightly.conf + └── weekly-offsite.conf /var/log/gniza/ # Log files ├── gniza-20260303-020000.log # Per-run logs -├── cron-nas.log # Per-remote cron output -└── cron-offsite.log +├── cron-nightly.log # Per-schedule cron output +└── cron-weekly-offsite.log ``` ## WHM Plugin -gniza includes a WHM plugin for managing backups through the cPanel/WHM web interface. +gniza includes a WHM plugin for managing backups through the cPanel/WHM web interface. All pages use **Tailwind CSS v4** with **DaisyUI v5** for styling. ### Installation @@ -330,7 +396,7 @@ When gniza is not yet configured (no remotes in `/etc/gniza/remotes.d/`), the da 1. **SSH Key** — Detects existing keys in `/root/.ssh/` (`id_ed25519`, `id_rsa`, `id_ecdsa`, `id_dsa`). Lets you select one or enter a custom path. Shows `ssh-keygen` and `ssh-copy-id` commands for creating new keys. -2. **Remote Destination** — Configure the first remote: name, host, port, user, SSH key (pre-filled from step 1), base path, bandwidth limit, and retention count. Creates a config file in `/etc/gniza/remotes.d/`. +2. **Remote Destination** — Configure the first remote: name, type (SSH/S3/GDrive), connection details, base path, bandwidth limit, and retention count. Tests the connection before saving. Creates a config file in `/etc/gniza/remotes.d/`. 3. **Schedule** — Optionally set a backup schedule (hourly/daily/weekly/monthly/custom) for the new remote. Installs the cron entry automatically. Can be skipped. @@ -341,16 +407,11 @@ The wizard is also accessible anytime from the dashboard quick links ("Run Setup | Page | URL | Description | |------|-----|-------------| | Dashboard | `index.cgi` | Overview, remote listing, cron status, quick links | +| Remotes | `remotes.cgi` | Add/edit/delete remote destinations (SSH/S3/GDrive) with connection testing | +| Schedules | `schedules.cgi` | Add/edit/delete schedules, per-schedule cron toggle | +| Restore | `restore.cgi` | Restore workflow: select account, remote, snapshot, then restore type (full/files/database/mailbox) | | Settings | `settings.cgi` | Edit main config (`/etc/gniza/gniza.conf`) | -| Remotes | `remotes.cgi` | Add/edit/delete remote destinations | -| Schedules | `schedules.cgi` | View and manage cron schedules | -| Setup Wizard | `setup.cgi` | Guided initial configuration | - -### SSH Key Guidance - -When adding a new remote (via Remotes > Add Remote), an SSH key guidance block is displayed above the form showing: -- Detected existing keys on the server -- Commands to generate a new key and copy it to the remote +| Setup Wizard | `setup.cgi` | Guided initial configuration (3 steps) | ### Plugin File Layout @@ -361,19 +422,23 @@ whm/ ├── index.cgi # Dashboard ├── setup.cgi # Setup wizard (3 steps) ├── settings.cgi # Main config editor - ├── remotes.cgi # Remote CRUD - ├── schedules.cgi # Cron management + ├── remotes.cgi # Remote CRUD (SSH/S3/GDrive) + ├── schedules.cgi # Schedule CRUD + cron toggles + ├── restore.cgi # Restore workflow (account → remote → snapshot → type) ├── assets/ - │ └── gniza-whm.css # Shared styles + │ ├── gniza-whm.css # Built Tailwind/DaisyUI CSS (committed) + │ └── src/ + │ ├── input.css # Tailwind v4 entry point + │ ├── safelist.html # Class safelist for Tailwind scanner + │ └── package.json # Build toolchain └── lib/GnizaWHM/ ├── Config.pm # Config parser/writer (pure Perl) ├── Validator.pm # Input validation - ├── Cron.pm # Cron read + allowlisted gniza commands - └── UI.pm # Navigation, flash, CSRF, HTML helpers + ├── Cron.pm # Cron read + per-schedule install/remove + ├── Runner.pm # Pattern-based safe CLI command runner + └── UI.pm # Navigation, flash, CSRF, HTML helpers, CSS delivery ``` -## Production Server - ## Running Tests ```bash