shutil and tempfile — Bulk File Ops and Temp Files

This article walks through two modules: shutil, which operates on whole files and folders, and tempfile, which handles disposable temporary files. You'll cover bulk file operations with shutil.copy / move / rmtree, the safe way to use NamedTemporaryFile, and the atomic write pattern that combines both.

shutil — File and Folder Operations in One Line

shutil stands for shell utilities — it's the standard library that lets you call shell-style operations like cp / mv / rm -r from Python in a single line. The os module has things like os.rename, but shutil adds higher-level operations such as recursive folder copy and tree delete.

import shutil
import os

# 1) Copy a file to another location (a typical backup)
os.makedirs("backups", exist_ok=True)        # Make sure the destination folder exists
shutil.copy("data/sales.csv", "backups/sales_2024.csv")

shutil.move("logs/temp.log", "logs/archive_2024.log")

shutil.rmtree("old_data")

rmtree Cannot Be Undone

shutil.rmtree(path) physically deletes everything under path. Python doesn't route through a trash bin — it's gone instantly. In a real project, print the target before deleting, add a guard like if path.startswith("/tmp"), and build habits that protect both you and your code.

Back up the sales CSV data/sales.csv as backups/sales_backup.csv. You can inspect the original from the 📂 Files panel on the left.

① Please import shutil and os.

② Please prepare the destination folder backups/ (in a way that doesn't error if it already exists).

③ Please copy the file to the destination name above.

④ Please verify both the backup and the original by printing Backup done: True / Source still exists: True.

(If you run it correctly the explanation appears below.)

tempfile — Safely Create Disposable Files

tempfile is the standard library for temporary files that are automatically cleaned up when you're done with them. You use it for storing intermediate results, buffering large data, holding test working folders — anything that only needs to exist while the process is running.

If you craft your own filenames like tmp_xxx.txt, you risk colliding with other processes or forgetting to delete them. tempfile instead creates files in the OS's temp folder under a guaranteed-unique name, and removes them automatically when the work is done.

import tempfile
import os

# Open safely with `with`, then auto-delete
with tempfile.NamedTemporaryFile(mode="w+", suffix=".txt", delete=True) as tf:
    tf.write("intermediate aggregation data\n")
    tf.seek(0)
    print("Content:", tf.read())
    print("Path:", tf.name)
# ← The block ends here and the file is auto-deleted

When to Combine delete=False with `with`

When you want to write the file and read it back later from somewhere else, pass delete=False so the file survives leaving the with block. Once you're done with it, either remove it yourself with os.remove(path) or swap it into the real destination with shutil.move — that's the atomic write pattern covered below.

Create a temporary file, write to it, then check whether it exists before and after deletion. Open a NamedTemporaryFile with delete=False, write to it, read it back, check existence before deletion, delete it, then check existence again.

① Please import tempfile and os.

② Please open a temp file with NamedTemporaryFile(mode="w", delete=False), write the string sales total: 3750 into it, save tf.name to a path variable, and exit the with block.

③ Please print existence with os.path.exists(path) in the form Exists before delete: ◯◯.

④ Please delete the file with os.unlink(path).

⑤ Please print existence again with os.path.exists(path) in the form Exists after delete: ◯◯.

Auto-Deleted Temp Files with `delete=True`

If you overwrite an important config file or state file directly with open("settings.json", "w"), a process crash mid-write can leave a half-written file behind, which then fails to parse on the next read. The standard way to avoid this is the "write to a temp file completely, then rename / move it into the real path" pattern — known as atomic write (atomic — even if interrupted, observers can only see "completed" or "not started", never an in-between state).

import tempfile
import shutil
import json

final_path = "settings.json"
new_settings = {"theme": "dark", "lang": "en", "fontSize": 14}

# 1) Write fully to a temp file
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".json") as tf:
    json.dump(new_settings, tf)
    temp_path = tf.name

# 2) Once finished, move into the real path (this single moment is the "swap")
shutil.move(temp_path, final_path)

# 3) Verify
with open(final_path) as f:
    print(json.load(f))
# → {'theme': 'dark', 'lang': 'en', 'fontSize': 14}

A file object has a current read/write position (cursor), and calling `tf.write(...)` moves the cursor right after what was written (= the end). Calling tf.read() from there returns an empty string because there's nothing past the cursor, so the standard move is to reset the cursor to the start with `tf.seek(0)` before reading.

mode="w+" Is Read-Write Mode

`mode="w+"` is the mode that lets you both write and read through the same file handle. With just "w" you're write-only and read() doesn't work; with just "r" you're read-only and write() doesn't work. For exercises that write and immediately read back, "w+" is required.

Create a temp file with `with NamedTemporaryFile(delete=True)` and confirm it's auto-deleted. Write to it inside the with block, read it back, then check existence after the block exits — it should be False.

① Please import tempfile and os.

② Please open tempfile.NamedTemporaryFile(mode="w+", delete=True) with a with statement (mode="w+" is read-write).

③ Inside the block, please write tf.write("config: ok\n"), reset the cursor to the start with tf.seek(0), read back with tf.read(), and print it as Content: ◯◯.

④ Please save the path inside the block as saved_path = tf.name.

⑤ After leaving the block, please print the result of os.path.exists(saved_path) in the form Exists after with: ◯◯ (it should be False).