new README

This commit is contained in:
Radek
2025-02-21 14:54:03 +00:00
parent c48b65a253
commit 383cd7e975
2 changed files with 26 additions and 75 deletions

View File

@@ -1,90 +1,41 @@
# **Azure Blob Sync Script - How To Use**
### **📌 How to Use**
This script syncs a directory to **Azure Blob Storage** using `azcopy` while:
**Avoiding business hours** (default: 9 AM - 5 PM, configurable).
**Preventing duplicate instances** (via a lock file).
**Automatically resuming** unfinished jobs.
**Logging progress & generating reports**.
#### **1⃣ Create the Directory List File**
---
Example: `/opt/AZURE/dir2sync_list`
## **📌 1. Script Usage**
Run the script manually:
```bash
./run_azcopy.sh <directory> [log=true|false] [bandwidth_mbps]
```
### **Example Commands**
- **Basic sync with no bandwidth limit:**
```bash
./run_azcopy.sh /opt/AZURE/ false
```
- **Enable logging & limit bandwidth to 10 Mbps:**
```bash
./run_azcopy.sh /opt/AZURE/ true 10
```
- **Run in the background & detach from terminal:**
```bash
nohup ./run_azcopy.sh /opt/AZURE/ true 10 & disown
```plaintext
/opt/AZURE/BLOB > testcontainer115
/opt/AZURE/OTHERBLOB > testcontainer115b
/opt/AZURE/EXTRABLOB > testcontainer_extra
```
---
## **⏲️ 2. Automating with Cron**
Schedule the script to **run every hour**:
#### **2⃣ Run the Script**
```bash
crontab -e
./run_azcopy.sh /opt/AZURE/dir2sync_list true 20
```
Add this line:
```bash
0 * * * * /path/to/run_azcopy.sh /opt/AZURE/ true 10
```
### **How It Works**
- Runs at **00 minutes past every hour** (e.g., 1:00, 2:00, 3:00, etc.).
- If already running, the **next cron execution exits** to prevent duplicates.
- If interrupted (e.g., business hours), the **next run resumes**.
- `true` → Enables logging.
- `20` → Limits bandwidth to **20 Mbps**.
---
## **🔍 3. Checking If the Script Is Running**
Check if `azcopy` or the script is running:
#### **3⃣ Cron Job (Runs Every Hour)**
```bash
pgrep -fl run_azcopy.sh
pgrep -fl azcopy
```
To **stop it manually**:
```bash
pkill -f azcopy
pkill -f run_azcopy.sh
0 * * * * /opt/AZURE/run_azcopy.sh /opt/AZURE/dir2sync_list true 20
```
---
## **📄 4. Checking Logs & Reports**
- **Sync Log (if enabled)**:
```bash
tail -f azcopy_log_*.txt
```
- **Completion Report**:
```bash
cat completion_report_*.txt
```
---
## **⚙️ 5. Customizing Business Hours**
Modify the script to change business hours:
```bash
BUSINESS_HOURS_START=9
BUSINESS_HOURS_END=17
```
---
## **✅ 6. Expected Behavior**
| Scenario | What Happens? |
|----------|--------------|
| **Cron runs script inside business hours** | Script exits immediately. |
| **Script is running when cron starts again** | Second instance exits to prevent duplicates. |
| **Sync job interrupted by business hours** | Next run resumes automatically. |
| **Sync completes normally** | Report logs all transferred files. |
### **✅ Key Features**
**Prevents multiple instances** using a lock file (`/tmp/run_azcopy.lock`).
**Reads a file with directory-to-container mappings.**
**Validates each source directory before syncing.**
**Checks if the destination container exists** (test write).
**Stops immediately on any error** (clean runs only).
**Monitors `azcopy` every 30 seconds** and stops if business hours begin.
**Processes all directories sequentially using an array (fixing loop issue).**
**Uses one log file for the entire run.**

0
run_azcopy-single-dir.sh Executable file → Normal file
View File