Commit Graph

40 Commits

Author SHA1 Message Date
shuki
04a0c45abc Add rsync auto-retry on partial transfer and show running tasks in header
Rsync exit 23 (partial transfer) now triggers an automatic retry to recover
failed files before accepting with a warning. Also adds a task counter next
to the clock in the header bar showing running job count.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 03:26:13 +02:00
shuki
fec13135ce Add source targets, docs panel, tail-style log viewer, and various improvements
- Add source.sh for remote source backup support
- Add responsive DocsPanel with layout adaptations for narrow screens
- Running tasks log viewer now shows last 100 lines (tail -f style)
- Add incremental backup explanation to README
- Update backup, transfer, schedule, and snaplog modules
- Add MCP config and logo asset

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 03:06:39 +02:00
shuki
19f6077e33 Fix progress stuck: use tee instead of read-line in _snaplog_tee
The old read -r line approach buffered until \n, but rsync --info=progress2
uses \r without \n for progress updates. Using tee(1) preserves \r
characters and writes through immediately for real-time progress.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:59:36 +02:00
shuki
0ea7d10a9c Add rsync progress bar to Running Tasks screen
- Add --info=progress2 --no-inc-recursive to rsync for overall progress
- Parse rsync progress output (percentage, speed, ETA) from log file
- Show ProgressBar widget and progress label below buttons
- Progress bar auto-hides when job finishes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:55:02 +02:00
shuki
ae1563396c Fix WORK_DIR override: config file was clobbering detect_mode
The settings screen wrote WORK_DIR="/usr/local/gniza/workdir" (root-mode
path) into gniza.conf. load_config then overrode the correct user-mode
path set by detect_mode, causing mktemp and rsync log redirects to fail
with Permission denied — crashing the bash script while rsync continued
as an orphan.

- Remove WORK_DIR from AppSettings model and settings screen
- Re-run detect_mode after config load to restore correct paths
- Removed stale WORK_DIR from user's gniza.conf

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:14:14 +02:00
shuki
3eb6c31509 Show rsync verbose output in TUI and application log
Replace tee with _snaplog_tee shell function that writes each line
unbuffered to three destinations: snapshot log, application log file,
and stderr (which the TUI captures for live display). This fixes the
issue where rsync file-by-file output was invisible in both the Logs
screen and the Running Tasks view due to pipe buffering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:33:15 +02:00
shuki
276b49ea0a Fix include filter: also include directory contents
When TARGET_INCLUDE has a directory pattern like 'embajada/', rsync
would include the directory but exclude its contents (files didn't
match any include rule before hitting --exclude=*). Now adds a
'pattern**' include for each directory pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:01:14 +02:00
shuki
28d85ed89f Add --sparse to rsync opts for efficient sparse file handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:52:17 +02:00
shuki
061ceac051 Add per-snapshot backup logs (dirvish-style)
Save log, rsync_error, summary, and index files into each snapshot
directory after backup. Rsync/rclone output is captured via tee during
transfer so the TUI still shows live output while logging to disk.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:48:01 +02:00
shuki
18af43936c Treat rsync exit codes 23/24 as warnings, not failures
Exit 23 (partial transfer, permission denied) and 24 (vanished files)
are expected in non-root backups. These no longer trigger retries or
fail the backup — they log a warning and continue.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:51:13 +02:00
shuki
feeea07b56 Fix cron backups failing due to minimal PATH environment
Cron runs with a very limited PATH that may not include sshpass, rsync,
etc. Add explicit PATH to generated cron lines and log output to
cron.log instead of /dev/null for debugging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:07:14 +02:00
shuki
e863d9a478 Fix shell injection in snapshot, retention, and restore remote_exec calls
Apply shquote() to all remaining remote_exec paths that interpolate
variables into single-quoted shell strings. Covers list/resolve/clean
snapshots, symlink updates, retention pruning, and restore file listing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:17:05 +02:00
shuki
63cc7f842e Fix security and correctness bugs found in code review
- Add shquote() to escape single quotes in paths passed to remote_exec,
  preventing shell injection via REMOTE_BASE containing single quotes
- Apply shquote to remote_exec calls in remotes.sh, backup.sh, transfer.sh, ssh.sh
- Add DISK_USAGE_THRESHOLD validation in config.sh
- Export SMTP_PASSWORD (was missing from export list)
- Fix WEB_PORT default mismatch: use 2323 consistently in from_conf and settings save
- Narrow exception catch in remotes.py disk info fetch to KeyError/LookupError
- Quote REMOTE_KEY in build_rsync_ssh_cmd for paths with spaces

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:13:20 +02:00
shuki
c68e93fc89 Fix disk info when remote base directory doesn't exist yet
Fall back to df / when REMOTE_BASE path doesn't exist on the remote.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:17:42 +02:00
shuki
0eb4dc00ed Fix remote_disk_info_short with robust df parsing
Parse df output locally using awk to find the % field and extract
the 3 size fields before it. Handles wrapped lines and CR chars.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:14:59 +02:00
shuki
ccae8451c3 Fix remote_disk_info_short — parse df on remote side
Move awk parsing to the remote to avoid SSH output encoding issues
that caused empty fields when parsing locally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:13:46 +02:00
shuki
279b05b5e4 Fix disk usage parsing — use grep instead of df --output
df --output=pcent is not available on all systems. Use grep -oP to
extract the percentage field from standard df output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:12:18 +02:00
shuki
604437bed2 Fix remote disk usage check returning empty percentage
Use df --output=pcent for reliable single-column output instead of
parsing multi-column df output through awk over SSH.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:11:18 +02:00
shuki
590b87843d Add configurable disk usage threshold setting
Disk usage threshold (default 95%) can now be controlled from
Settings. Set to 0 to disable the check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:08:42 +02:00
shuki
993a66d8c6 Add include/exclude filter support and disk space pre-check
- Add TARGET_INCLUDE field for rsync include patterns (comma-separated)
- Pass TARGET_INCLUDE and TARGET_EXCLUDE to rsync in transfer_folder
- Include mode uses --include='*/' + patterns + --exclude='*' + --prune-empty-dirs
- Abort backup if remote disk usage >= 95%

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:04:52 +02:00
shuki
eb9dda416b Remove Speed Test and Disk Info buttons from remotes screen
Disk info is now shown directly in the table. Speed test and detailed
disk info buttons/commands are no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:47:05 +02:00
shuki
8b3d6ede4f Show disk usage in remotes table with async loading
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:44:27 +02:00
shuki
c70fb1991c Add Disk Info and Speed Test buttons to Remotes screen
- Disk Info: runs df -h and df -i on remote via SSH (or locally)
- Speed Test: uploads/downloads 10MB test file via rsync, measures Mbps
- Both available as CLI commands: gniza remotes disk-info/speed-test --name=NAME

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:38:15 +02:00
shuki
c1d00108ba Add MySQL grants backup and MySQL restore support
- mysql_dump_grants(): backs up user grants via SHOW CREATE USER + SHOW GRANTS,
  skipping system users (root, mysql.sys, etc.)
- mysql_restore_databases(): restores .sql.gz dumps and grants.sql from snapshot
- Backup flow: grants dumped alongside database dumps into _mysql/
- Restore flow: automatically restores MySQL databases and grants when _mysql/
  exists in snapshot and target has MySQL enabled
- CLI: --skip-mysql flag to opt out of MySQL restore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 04:53:41 +02:00
shuki
1425c416eb Add per-schedule active toggle with crontab sync
Each schedule has SCHEDULE_ACTIVE field (yes/no). Table shows active
status with checkmark/cross. Toggle Active button flips state and
reinstalls crontab with only active schedules. Inactive schedules
are skipped during crontab install.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 04:38:51 +02:00
shuki
313d1fe873 Replace free-text day inputs with proper dropdowns per schedule type
- Daily: multi-select checkboxes for days of week (empty = every day)
- Weekly: single dropdown for day of week
- Monthly: dropdown with 1st/7th/14th/21st/28th of the month
- Cron generation supports daily with specific days via SCHEDULE_DAY

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 03:11:49 +02:00
shuki
b3c055c9be Add configurable WORK_DIR instead of /tmp for temp files
Default: /usr/local/gniza/workdir (root) or ~/.local/state/gniza/workdir
(user). MySQL dumps and rclone temp configs now use WORK_DIR. Configurable
via gniza.conf or TUI Settings screen. Created during install.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 03:00:21 +02:00
shuki
791584aa60 Fix MySQL dump remote path to use _mysql/ instead of full temp path
Add optional dest_name parameter to transfer_folder to override the
remote subpath. MySQL dumps now land in _mysql/ inside the snapshot
instead of tmp/gniza-mysql-xxx/_mysql/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 02:56:37 +02:00
shuki
733bb5de62 Add MySQL database backup support
Dump MySQL/MariaDB databases as part of backup snapshots. Each database
is dumped to a separate gzipped file under _mysql/ in the snapshot.
Supports all-databases mode with exclude list, or explicit database
selection. Includes TUI form fields for full configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 02:48:47 +02:00
shuki
121a615e67 Remove legacy gum TUI and clean up install script
- Delete all lib/ui_*.sh files (gum-based TUI)
- Remove gum download from install script
- Remove gum fallback and show_logo from bin/gniza
- Update README to reference Textual TUI and web GUI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 02:15:52 +02:00
shuki
3d5a9cdfdd Remove verify feature and align main menu layout
Remove verify screen, CLI subcommand, and all related functions.
Fix logo/menu height alignment on main menu screen.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 01:57:55 +02:00
shuki
611fe7da3f Add test connection step when creating remotes
- SSH: tests ssh connection before saving, option to save anyway on failure
- Local: checks if directory exists
- S3: tests rclone connection if rclone is available
- GDrive: tests rclone connection if rclone is available
- All tests are optional and can be skipped

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:57:57 +02:00
shuki
74efc08903 Trigger wizard when either remote or target is missing
- Changed condition from AND to OR: wizard runs if remotes OR targets
  are not configured (not only when both are empty)
- Wizard skips steps that are already done (e.g. if remote exists,
  jumps straight to target creation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:47:56 +02:00
shuki
4306b9e3b3 Open file browser immediately in folder picker
- First gum file browser opens right after target name entry
- Shows selected folders list, then offers Add/Remove/Done
- No extra menu step before the first folder selection

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:43:45 +02:00
shuki
4f9ab94c69 Clarify target name prompt mentions file browser next
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:41:24 +02:00
shuki
a956bd9e30 Fix gum file browser returning relative paths
gum file returns paths relative to the start directory.
Prepend / to ensure absolute paths for backup targets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:29:30 +02:00
shuki
f80c786597 Use gum file browser for folder selection in target picker
Replace manual path input with gum file --directory for browsing
and selecting folders. Adds duplicate detection. Shows selected
count in menu header.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 22:22:33 +02:00
shuki
6c16f65c3c Replace whiptail TUI with gum (charmbracelet/gum)
- Rewrite ui_common.sh wrappers to use gum choose, gum input,
  gum confirm, gum style, gum pager
- Menu items display as "TAG  Description" for reliable tag extraction
  (handles duplicate descriptions safely)
- Replace whiptail --gauge with text-based progress bar
- Use printf %b instead of echo -e to avoid escape injection
- Read msgbox keypress from /dev/tty to work in piped contexts
- Update bin/gniza, install.sh, README.md references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 21:45:15 +02:00
shuki
0004dbed9f Add first-time setup wizard for new installations
Guides users through creating their first remote and target when gniza
launches with no configuration. Optionally runs a first backup.
Triggers only when both remotes.d/ and targets.d/ are empty.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 21:38:19 +02:00
shuki
928d5af54c Initial implementation of gniza4linux backup tool
Complete Linux backup manager with Whiptail TUI and CLI interface.
Adapted from gniza4cp (cPanel backup tool) with target/profile-based
system replacing cPanel-specific features.

- 14 core engine modules (backup, restore, targets, remotes, transfer, etc.)
- 11 Whiptail TUI screens (full CRUD for targets/remotes/schedules)
- CLI entrypoint with subcommands for scripting/cron
- Support for SSH, local, S3, and Google Drive remotes
- rsync --link-dest incremental snapshots
- Root and user mode (XDG paths)
- 70 passing tests
- Config templates, installer, uninstaller

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 21:15:29 +02:00