new README

This commit is contained in:
Radek
2025-02-21 14:54:03 +00:00
parent c48b65a253
commit 383cd7e975
2 changed files with 26 additions and 75 deletions

View File

@@ -1,90 +1,41 @@
# **Azure Blob Sync Script - How To Use** ### **📌 How to Use**
This script syncs a directory to **Azure Blob Storage** using `azcopy` while: #### **1⃣ Create the Directory List File**
**Avoiding business hours** (default: 9 AM - 5 PM, configurable).
**Preventing duplicate instances** (via a lock file).
**Automatically resuming** unfinished jobs.
**Logging progress & generating reports**.
--- Example: `/opt/AZURE/dir2sync_list`
## **📌 1. Script Usage** ```plaintext
Run the script manually: /opt/AZURE/BLOB > testcontainer115
```bash /opt/AZURE/OTHERBLOB > testcontainer115b
./run_azcopy.sh <directory> [log=true|false] [bandwidth_mbps] /opt/AZURE/EXTRABLOB > testcontainer_extra
```
### **Example Commands**
- **Basic sync with no bandwidth limit:**
```bash
./run_azcopy.sh /opt/AZURE/ false
```
- **Enable logging & limit bandwidth to 10 Mbps:**
```bash
./run_azcopy.sh /opt/AZURE/ true 10
```
- **Run in the background & detach from terminal:**
```bash
nohup ./run_azcopy.sh /opt/AZURE/ true 10 & disown
``` ```
--- ---
## **⏲️ 2. Automating with Cron** #### **2⃣ Run the Script**
Schedule the script to **run every hour**:
```bash ```bash
crontab -e ./run_azcopy.sh /opt/AZURE/dir2sync_list true 20
``` ```
Add this line:
```bash - `true` → Enables logging.
0 * * * * /path/to/run_azcopy.sh /opt/AZURE/ true 10 - `20` → Limits bandwidth to **20 Mbps**.
```
### **How It Works**
- Runs at **00 minutes past every hour** (e.g., 1:00, 2:00, 3:00, etc.).
- If already running, the **next cron execution exits** to prevent duplicates.
- If interrupted (e.g., business hours), the **next run resumes**.
--- ---
## **🔍 3. Checking If the Script Is Running** #### **3⃣ Cron Job (Runs Every Hour)**
Check if `azcopy` or the script is running:
```bash ```bash
pgrep -fl run_azcopy.sh 0 * * * * /opt/AZURE/run_azcopy.sh /opt/AZURE/dir2sync_list true 20
pgrep -fl azcopy
```
To **stop it manually**:
```bash
pkill -f azcopy
pkill -f run_azcopy.sh
``` ```
--- ---
## **📄 4. Checking Logs & Reports** ### **✅ Key Features**
- **Sync Log (if enabled)**: **Prevents multiple instances** using a lock file (`/tmp/run_azcopy.lock`).
```bash **Reads a file with directory-to-container mappings.**
tail -f azcopy_log_*.txt **Validates each source directory before syncing.**
``` **Checks if the destination container exists** (test write).
- **Completion Report**: **Stops immediately on any error** (clean runs only).
```bash **Monitors `azcopy` every 30 seconds** and stops if business hours begin.
cat completion_report_*.txt **Processes all directories sequentially using an array (fixing loop issue).**
``` **Uses one log file for the entire run.**
---
## **⚙️ 5. Customizing Business Hours**
Modify the script to change business hours:
```bash
BUSINESS_HOURS_START=9
BUSINESS_HOURS_END=17
```
---
## **✅ 6. Expected Behavior**
| Scenario | What Happens? |
|----------|--------------|
| **Cron runs script inside business hours** | Script exits immediately. |
| **Script is running when cron starts again** | Second instance exits to prevent duplicates. |
| **Sync job interrupted by business hours** | Next run resumes automatically. |
| **Sync completes normally** | Report logs all transferred files. |

0
run_azcopy-single-dir.sh Executable file → Normal file
View File