Commit Graph

144 Commits

Author SHA1 Message Date
shuki
fdf2f0b10a Show job log inline below running tasks table
View Log now displays the log in a RichLog panel below the buttons
instead of opening a modal screen. The log tails the file in real-time
with a 0.3s poll interval while the job is running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:47:49 +02:00
shuki
83ccf44117 Fix live log not updating: tail log file directly in OperationLog
The poll timer was reading from job.output in memory, which depended on
run_job's async task populating it. Now OperationLog reads new content
directly from the log file using seek, making it independent of the
async task. Also store the asyncio task reference to prevent GC.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:40:56 +02:00
shuki
d60c2f06aa Add Status column to Logs screen
Detects status from log content: Success (backup completed, no errors),
Failed (ERROR/FATAL markers), OK (lock released), Interrupted (no clean exit).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:34:42 +02:00
shuki
ad65a376fd Use subprocess.Popen for background jobs to survive TUI exit
asyncio's SubprocessTransport sends SIGKILL to child processes during
event loop cleanup, killing the gniza bash wrapper when the TUI exits.
Switch to subprocess.Popen which has no such cleanup behavior, allowing
backup jobs to continue running after the TUI is closed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:28:32 +02:00
shuki
ae1563396c Fix WORK_DIR override: config file was clobbering detect_mode
The settings screen wrote WORK_DIR="/usr/local/gniza/workdir" (root-mode
path) into gniza.conf. load_config then overrode the correct user-mode
path set by detect_mode, causing mktemp and rsync log redirects to fail
with Permission denied — crashing the bash script while rsync continued
as an orphan.

- Remove WORK_DIR from AppSettings model and settings screen
- Re-run detect_mode after config load to restore correct paths
- Removed stale WORK_DIR from user's gniza.conf

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:14:14 +02:00
shuki
58819abcd4 Persist finished jobs in registry and fix reconnection bugs
- Save finished jobs to registry (24h TTL) so they survive TUI restart
- Fix PermissionError in PID check incorrectly marking alive processes as dead
- Handle CancelledError explicitly to preserve running status on TUI exit
- Tail log files for reconnected running jobs instead of showing stale output
- Detect actual return code from log content; show "?" for unknown status

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 21:06:48 +02:00
shuki
8fec087987 Write job output to file instead of pipe to survive TUI exit
The subprocess stdout was a pipe to the TUI. When the TUI exited, the
pipe broke (SIGPIPE) and killed the backup process. Now the subprocess
writes to a log file in WORK_DIR, and the TUI tails it for live
display. When the TUI exits, the subprocess keeps running because it
writes to a file, not a pipe. On restart, the log file is loaded to
show output for reconnected or finished background jobs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:50:09 +02:00
shuki
a08cf9911c Show background jobs that finished while TUI was closed
Previously _load_registry silently dropped dead PIDs, so jobs that
completed in the background were invisible on restart. Now dead PIDs
are loaded as 'success' status so the user sees they completed. The
registry is cleaned up after loading.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:46:47 +02:00
shuki
0de0f7783a Don't kill running jobs on TUI exit
Previously action_quit() killed all running jobs before exiting,
making it impossible to reconnect to them on restart. Now shows a
confirmation dialog when jobs are running and lets them continue
in the background.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 20:38:03 +02:00
shuki
3eb6c31509 Show rsync verbose output in TUI and application log
Replace tee with _snaplog_tee shell function that writes each line
unbuffered to three destinations: snapshot log, application log file,
and stderr (which the TUI captures for live display). This fixes the
issue where rsync file-by-file output was invisible in both the Logs
screen and the Running Tasks view due to pipe buffering.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:33:15 +02:00
shuki
ba93a74f8e Persist running jobs to registry for TUI reconnection
Jobs are now saved to gniza-jobs.json in WORK_DIR when they start and
finish. On TUI restart, the registry is loaded and PIDs are checked —
still-running jobs appear in the Running Tasks screen and can be
killed. Reconnected jobs are polled every second to detect completion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:27:37 +02:00
shuki
276b49ea0a Fix include filter: also include directory contents
When TARGET_INCLUDE has a directory pattern like 'embajada/', rsync
would include the directory but exclude its contents (files didn't
match any include rule before hitting --exclude=*). Now adds a
'pattern**' include for each directory pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 19:01:14 +02:00
shuki
28d85ed89f Add --sparse to rsync opts for efficient sparse file handling
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:52:17 +02:00
shuki
061ceac051 Add per-snapshot backup logs (dirvish-style)
Save log, rsync_error, summary, and index files into each snapshot
directory after backup. Rsync/rclone output is captured via tee during
transfer so the TUI still shows live output while logging to disk.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:48:01 +02:00
shuki
92828ab891 Debug: show row key and job IDs when kill fails to find job
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:24:15 +02:00
shuki
53476455e3 Improve kill job feedback messages
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:22:55 +02:00
shuki
54c7b75cee Add debug output to kill_job to diagnose kill failure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:20:53 +02:00
shuki
0110b00b67 Decouple job lifecycle from screen workers
Jobs were tied to screen @work tasks, so switch_screen cancelled the
worker and lost the process reference. Now start_job uses
asyncio.create_task so jobs survive screen changes and can be killed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:18:39 +02:00
shuki
6a0389f437 Use SIGKILL and os.getpgid for reliable job killing
SIGTERM was being ignored by child processes. Use SIGKILL via the
actual process group ID, with proc.kill() as fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:16:09 +02:00
shuki
5e21d1b20a Navigate to Running Tasks after starting backup or restore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:15:30 +02:00
shuki
df07e4e4f9 Remove duplicate completion notifications from backup/restore screens
The app-level on_job_finished handler already shows these.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:12:33 +02:00
shuki
c96930f3ff Kill entire process group when stopping a job
Start CLI subprocesses in their own session so SIGTERM via
os.killpg reaches child processes (rsync, etc.) not just the shell.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:11:37 +02:00
shuki
2379c2fdeb Add Kill Job button to Running Tasks screen
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:10:19 +02:00
shuki
8a83812584 Add background jobs system with Running Tasks screen
Backup and restore operations now run as background jobs instead of
blocking modal screens. Users can navigate away and check progress
from a dedicated Running Tasks screen. OperationLog supports attaching
to running jobs with live output polling.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:07:34 +02:00
shuki
18af43936c Treat rsync exit codes 23/24 as warnings, not failures
Exit 23 (partial transfer, permission denied) and 24 (vanished files)
are expected in non-root backups. These no longer trigger retries or
fail the backup — they log a warning and continue.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:51:13 +02:00
shuki
3e4b906f7e Only create log files for backup/restore/retention commands
Utility commands (schedule, snapshots, remotes, etc.) were creating
empty 0 B log files on every invocation. Now init_logging only runs
for commands that produce meaningful log output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:48:14 +02:00
shuki
0b0204dada Format log filenames as readable date/time columns in logs screen
Show '2026-03-06' and '14:45:16' instead of 'gniza-20260306-144516.log'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:46:10 +02:00
shuki
f9f7ae6ad3 Format snapshot timestamps as readable dates in snapshots browser
Display '2026-03-06 14:07:06' instead of raw '2026-03-06T140706'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:42:41 +02:00
shuki
99774efe52 Show 'never' instead of '--' for empty last/next run in schedule table
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:09:55 +02:00
shuki
feeea07b56 Fix cron backups failing due to minimal PATH environment
Cron runs with a very limited PATH that may not include sshpass, rsync,
etc. Add explicit PATH to generated cron lines and log output to
cron.log instead of /dev/null for debugging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 16:07:14 +02:00
shuki
04044eda83 Fix installer exit when WEB_USER not found in config
grep returns exit 1 when no match, which with set -eo pipefail kills
the script. Add || true to the grep pipeline in credential extraction.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:40:25 +02:00
shuki
5d8764ad25 Fix installer crash when enabling web dashboard as non-root user
The grep -v pipeline combined with set -eo pipefail caused the entire
installer to exit silently when the user answered "y" to web dashboard.
Replace with proper if/else and add user-level systemd service support
(systemctl --user) for non-root installs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 15:38:28 +02:00
shuki
e863d9a478 Fix shell injection in snapshot, retention, and restore remote_exec calls
Apply shquote() to all remaining remote_exec paths that interpolate
variables into single-quoted shell strings. Covers list/resolve/clean
snapshots, symlink updates, retention pruning, and restore file listing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:17:05 +02:00
shuki
63cc7f842e Fix security and correctness bugs found in code review
- Add shquote() to escape single quotes in paths passed to remote_exec,
  preventing shell injection via REMOTE_BASE containing single quotes
- Apply shquote to remote_exec calls in remotes.sh, backup.sh, transfer.sh, ssh.sh
- Add DISK_USAGE_THRESHOLD validation in config.sh
- Export SMTP_PASSWORD (was missing from export list)
- Fix WEB_PORT default mismatch: use 2323 consistently in from_conf and settings save
- Narrow exception catch in remotes.py disk info fetch to KeyError/LookupError
- Quote REMOTE_KEY in build_rsync_ssh_cmd for paths with spaces

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:13:20 +02:00
shuki
1c21e7a9df Update docs for include/exclude filters, MySQL, disk threshold, web dashboard
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 08:01:14 +02:00
shuki
c68e93fc89 Fix disk info when remote base directory doesn't exist yet
Fall back to df / when REMOTE_BASE path doesn't exist on the remote.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:17:42 +02:00
shuki
0eb4dc00ed Fix remote_disk_info_short with robust df parsing
Parse df output locally using awk to find the % field and extract
the 3 size fields before it. Handles wrapped lines and CR chars.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:14:59 +02:00
shuki
ccae8451c3 Fix remote_disk_info_short — parse df on remote side
Move awk parsing to the remote to avoid SSH output encoding issues
that caused empty fields when parsing locally.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:13:46 +02:00
shuki
279b05b5e4 Fix disk usage parsing — use grep instead of df --output
df --output=pcent is not available on all systems. Use grep -oP to
extract the percentage field from standard df output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:12:18 +02:00
shuki
604437bed2 Fix remote disk usage check returning empty percentage
Use df --output=pcent for reliable single-column output instead of
parsing multi-column df output through awk over SSH.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:11:18 +02:00
shuki
590b87843d Add configurable disk usage threshold setting
Disk usage threshold (default 95%) can now be controlled from
Settings. Set to 0 to disable the check.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:08:42 +02:00
shuki
993a66d8c6 Add include/exclude filter support and disk space pre-check
- Add TARGET_INCLUDE field for rsync include patterns (comma-separated)
- Pass TARGET_INCLUDE and TARGET_EXCLUDE to rsync in transfer_folder
- Include mode uses --include='*/' + patterns + --exclude='*' + --prune-empty-dirs
- Abort backup if remote disk usage >= 95%

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 07:04:52 +02:00
shuki
eb9dda416b Remove Speed Test and Disk Info buttons from remotes screen
Disk info is now shown directly in the table. Speed test and detailed
disk info buttons/commands are no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:47:05 +02:00
shuki
8b3d6ede4f Show disk usage in remotes table with async loading
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:44:27 +02:00
shuki
c70fb1991c Add Disk Info and Speed Test buttons to Remotes screen
- Disk Info: runs df -h and df -i on remote via SSH (or locally)
- Speed Test: uploads/downloads 10MB test file via rsync, measures Mbps
- Both available as CLI commands: gniza remotes disk-info/speed-test --name=NAME

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:38:15 +02:00
shuki
f7c5836db0 Show system clock in top right corner of all screens
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:35:51 +02:00
shuki
5c1fa04657 Add Last Run and Next Run columns to Schedules screen
- Last Run: timestamp of most recent backup log file
- Next Run: calculated from schedule type, time, and day settings
- Handles hourly, daily, weekly, monthly schedule types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:31:49 +02:00
shuki
220d30e515 Change default web port from 8080 to 2323
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:27:16 +02:00
shuki
907c77feb4 Remove username prompt, default to admin (configurable in gniza.conf)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:26:16 +02:00
shuki
343b49d245 Move web credentials to post-install summary section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 06:25:08 +02:00