Python OS Module Hacks You Probably Haven’t Tried

The Python os module is a handy tool for working with your computer’s operating system directly from your Python code. It offers functions to handle tasks like managing files and directories, manipulating processes, and controlling environment variables.

One of the great things about the os module is that it works on pretty much any operating system you can think of – Windows, macOS, Linux, you name it. This means you can write your code once and run it anywhere without worrying about compatibility issues.

If you want to learn how to manipulate file paths with Python, explore the tutorial on the os.path module.

In this tutorial, we’ll learn about the OS module in Python. Even if you’re new to programming, I’ll make it easy to understand. By the end, you’ll be able to use Python to manage your computer’s files like a pro!

So, let’s get started exploring the Python OS module together. It’ll be fun and useful, see what you’ll learn in this tutorial.

How To Create Directories using os Module

Creating directories (also known as folders) is a common task when working with files on your computer. Python’s os module provides two main functions for creating directories: os.mkdir() and os.makedirs().

Here’s how to use each of them:

The os.mkdir() function is used to create a single directory. Here’s how you can use it:

import os

# Specify the name of the directory you want to create
directory_name = "new_directory"

# Use os.mkdir() to create the directory
os.mkdir(directory_name)

The os.makedirs() function is similar to os.mkdir(), but it allows you to create multiple directories at once, including parent directories if they don’t already exist. Here’s how you can use it:

import os

# Specify the full path of the directory structure you want to create
directory_path = "parent_directory/subdirectory/subsubdirectory"

# Create the directory structure
os.makedirs(directory_path)

Here’s a complete practical example to create a directory and directory structure in your current working folder.

Example:

import os

# Print the list of files and directories in the current working directory
print(os.listdir())

# Create a new directory named "sampledir"
os.mkdir("sampledir")

# Create a directory structure with nested directories:
# - "sample2dir" containing a subdirectory "subdir" containing a directory "dir1"
os.makedirs("sample2dir/subdir/dir1")

# Create another directory structure with nested directories:
# - "sample3dir" containing a subdirectory "subdir1" containing a directory "dir2"
os.makedirs("sample3dir/subdir1/dir2")

# Print the list of files and directories in the current working directory after creating directories
print(os.listdir())

Output:

Before using the make directory functions :
['file 2.txt', 'file 3.txt', 'file.txt', 'os.py']
After using the make directory functions :
['file 2.txt', 'file 3.txt', 'file.txt', 'os.py', 'sample2dir', 'sample3dir', 'sampledir']

How To Remove Files And Directories using os Module

The os module provides convenient functions for removing files and directories from your file system. You can use these functions – os.rmdir(), os.removedirs(), and os.remove().

The os.rmdir() function serves to remove an empty directory. Simply provide the path of the directory you wish to delete as an argument. If the directory contains any files or subdirectories, os.rmdir() will raise an error.

import os

# Remove an empty directory named "my_directory"
os.rmdir("my_directory")

For removing a directory and its parent directories, if they become empty after deletion, you can utilize the os.removedirs() function. It operates similarly to os.rmdir() but accounts for potential parent directory removal.

import os

# Remove a directory and all its contents
os.removedirs("parent_directory/subdirectory")

When the goal is to delete a single file, the os.remove() function is the tool of choice. You have to provide the path of the file to be deleted as the function’s argument.

import os

# Remove a file named "example.txt"
os.remove("example.txt")

Now let’s use these functions to remove folders and files in our current working directory.

Example:

import os 

print("Before using the remove directory and file functions:")
print(os.listdir()) 

# remove directory and file functions of the os module
os.rmdir("sampledir")
os.removedirs("sample2dir/subdir/dir1")
os.remove("file 2.txt")

print("After using the remove directory and file functions:")
print(os.listdir())

Output:

Before using the remove directory and file functions:
['file 2.txt', 'file 3.txt', 'file.txt', 'os.py', 'sample2dir', 'sample3dir', 'sampledir']
After using the remove directory and file functions:
['file 3.txt', 'file.txt', 'os.py', 'sample3dir']

How To Rename Files And Directories using os Module

Renaming files and directories manually can be time-consuming, especially when dealing with large numbers of files or complex directory structures. You can streamline this process and improve efficiency by using Python’s os module. It provides you with two functions specifically for this purpose: os.rename() and os.renames().

The os.rename() function allows you to rename a single file or directory. Here’s how you can use it:

import os

# Specify the current and new names of the file or directory
old_name = "old_name.txt"
new_name = "new_name.txt"

# Rename the file or directory
os.rename(old_name, new_name)

If you need to rename a file or directory and also update its parent directories as necessary, you can use the os.renames() function. This function is particularly useful when dealing with nested directory structures. Here’s how it works:

import os

# Specify the current and new paths of the file or directory
old_path = "old_directory/old_file.txt"
new_path = "new_directory/new_file.txt"

# Rename the file or directory, updating parent directories as needed
os.renames(old_path, new_path)

Let’s see these functions in action:

Example:

import os 

print("Before using the rename functions :")
print(os.listdir()) 

os.rename('file 3.txt','file34.txt')
os.renames('sample3dir/subdir1/dir2','sample3renameddir/subdir1renamed/dir2renamed')

print("After using the rename functions :")
print(os.listdir()) 

Output:

Before using the rename functions :
['file 3.txt', 'file.txt', 'os.py', 'sample3dir']
After using the rename functions :
['file.txt', 'file34.txt', 'os.py', 'sample3renameddir']

How To Replace A File And Directory using os Module

Replacing a file or directory in Python is straightforward with the os module’s os.replace() function. This function helps you swap out an existing file or directory with a new one, simplifying the process of updating or overwriting files and directories.

Example:

import os 

print("Before using the replace function :")
print(os.listdir()) 

os.replace('file34.txt','file.txt')

print("After using the replace function :")
print(os.listdir()) 

Output:

Before using the replace function :
['file.txt', 'file34.txt', 'os.py', 'sample3renameddir']
After using the replace function :
['file.txt', 'os.py', 'sample3renameddir']

Listing Files and Folders

If you want to know what files and folders are in a certain place on your computer, there’s a handy tool called os.listdir(). This function lets you get a list of all the files and directories within a specified location on your computer. If you don’t specify a path, it’ll show you what’s in the current working directory.

Example:

import os
print(os.listdir())
print(os.listdir('sampledir'))

Output:

['file.txt', 'file2.txt', 'file3.txt', 'os.py', 'sampledir']
['file4.txt', 'file5.txt', 'subdir']

Changing Working Directory And Knowing Your Current Location

If you want to navigate the file system of your computer using Python, the os module provides functions to manage the working directory, allowing you to change the current directory and retrieve its path. The primary functions for these tasks are os.chdir(), os.getcwd(), and os.getcwdb().

Sometimes you need to work in a specific folder. For instance, if your Python script needs to access files or resources in a particular directory, you can use os.chdir() to change the current working directory.

Here’s how you do it:

import os

# Specify the path of the directory you want to work in
new_directory = "/path/to/new/directory"

# Change the working directory
os.chdir(new_directory)

To find out which directory your Python script is currently operating in, you can use os.getcwd(). This function returns the full path of the current working directory as a string.

In some cases, you may need the current working directory represented as bytes instead of a string. This is where os.getcwdb() comes in handy. It returns the current working directory as a bytes object.

Example:

import os

print("Before changing the directory :")
print(os.getcwd())
print(os.getcwdb())
print(os.listdir()) 

os.chdir("sampledir")

print("After changing the directory :")
print(os.getcwd())
print(os.getcwdb())
print(os.listdir())

Output:

Before changing the directory :
E:\blog writing\os example program\osfolder
b'E:\\blog writing\\os example program\\osfolder'
['file.txt', 'file2.txt', 'file3.txt', 'os.py', 'sampledir']
After changing the directory :
E:\blog writing\os example program\osfolder\sampledir
b'E:\\blog writing\\os example program\\osfolder\\sampledir'
['file4.txt', 'file5.txt', 'subdir']

Identifying the User

In Python, the os module provides a useful function called os.getlogin() to identify the currently logged-in user on the system. This function tells you the name of the user who’s logged in and running the Python script.

Imagine you’re writing a program that saves files in the user’s personal folder, knowing the current user’s name helps you find the right folder.

os.getlogin() might behave differently on different computers or operating systems. It might not work the same way everywhere.

Example:

import os
print("os.getlogin() returns you the user name of the system :")
print(os.getlogin())

Output:

os.getlogin() returns you the user name of the system :
well

Navigating Directory Trees with OS.walk()

The os.walk() function helps you explore all the folders and files within a directory and its subdirectories. To use this, first, you have to specify a starting directory (referred to as top argument) and Python goes through all the folders and files within it, including subdirectories.

It yields a tuple for each folder it encounters, containing three values:

• the folder’s name.
• the names of any subfolders inside it.
• and the names of all the files it contains.

By using this function, you can perform various operations on directory contents, such as searching for specific files, analyzing directory structures, or performing actions on each file or directory encountered.

Example:

import os

print("This is the result of os.walk() :")
print(list(os.walk("sampledir")))

Output:

This is the result of os.walk() :
[('sampledir', ['subdir'], ['file4.txt', 'file5.txt']), ('sampledir\\subdir', [], [])

You can also determine the order in which os.walk() traverses the directory tree by using the topdown argument. If topdown is set to False, os.walk() traverses the directory tree from bottom to top, meaning it explores the contents of directories (subdirectories and files) before the directories themselves. If you don’t specify, it defaults to True.

Example:

import os

print("Top-down approach of os.walk() :")
for i in os.walk("sampledir",topdown=1):
    print(i)

print("Bottom-up approach of os.walk() :")
for i in os.walk("sampledir",topdown=0):
    print(i)

Output:

Top-down approach of os.walk() :
('sampledir', ['subdir'], ['file4.txt', 'file5.txt'])
('sampledir\\subdir', [], [])
Bottom-up approach of os.walk() :
('sampledir\\subdir', [], [])
('sampledir', ['subdir'], ['file4.txt', 'file5.txt'])

Exploring Directories Efficiently with os.scandir()

When you use os.scandir(), it doesn’t just show you the names of files and folders. For each item it finds, it hands you an os.DirEntry object named entry, and you can then use this object to get information about each file or folder, like its name, size, or when it was last modified.

os.DirEntry objects provide access to various attributes and methods that allow you to gather information about the entry and perform operations on it.

Some of the most commonly used attributes and methods include:

name : The name of the entry (file or directory) within the directory.

path : The full path to the entry, including the directory it resides in.

is_dir() : A method that returns True if the entry is a directory, and False otherwise.

is_file() : A method that returns True if the entry is a regular file, and False otherwise.

stat() : A method that returns an os.stat_result object containing file information such as size, permissions, and timestamps.

You only need to provide one argument, path, which is the directory you want to scan. If you don’t specify any path, it will scan the current working directory.

Example 1:

import os

print(list(os.scandir()))
print(list(os.scandir("sampledir")))

Output:

[<DirEntry 'file.txt'>, <DirEntry 'file2.txt'>, <DirEntry 'file3.txt'>, <DirEntry 'os.py'>, <DirEntry 'sampledir'>]
[<DirEntry 'file4.txt'>, <DirEntry 'file5.txt'>, <DirEntry 'subdir'>]

Example 2:

import os

for i in os.scandir():
    print(f"{i} \nname: {i.name}\npath: {i.path}\ninode: {i.inode()}\nis this dir: {i.is_dir()}\nis this file: {i.is_file()}\n")

Output:

<DirEntry 'file.txt'>
name: file.txt
path: .\file.txt
inode: 1688849860316146
is this dir: False
is this file: True

<DirEntry 'file2.txt'>
name: file2.txt
path: .\file2.txt
inode: 1970324837026830
is this dir: False
is this file: True

<DirEntry 'file3.txt'>
name: file3.txt
path: .\file3.txt
inode: 1970324837026831
is this dir: False
is this file: True

<DirEntry 'os.py'>
name: os.py
path: .\os.py
inode: 1407374883604891
is this dir: False
is this file: True

<DirEntry 'sampledir'>
name: sampledir
path: .\sampledir
inode: 1970324837026829
is this dir: True
is this file: False

File Operations with Python’s OS Module

You can also read and write files using the os module.

While the OS module offers some file manipulation functions, Python’s built-in open() function is generally more convenient and commonly used. However, understanding these OS functions can be valuable in certain advanced scenarios or when working with file descriptors directly. So, now let’s explore the os.open() function.

The os.open() function provides a lower-level approach to open a file and you can also set various flags that determine the file’s mode. Flags are constants that help you decide whether you’re opening the file for reading, writing, both, or appending data. It returns a unique identifier called a file descriptor(fd), which represents an open file in the operating system.

Some Mostly Used Flags:

• os.O_RDONLY – Opens the file for reading only.
• os.O_WRONLY – Opens the file for writing only.
• os.O_RDWR – Opens the file for both reading and writing.
• os.O_APPEND – Opens the file for appending data.

Reading and Writing Bytes

Once a file is open and you have its file descriptor, you can write a byte string to the file associated with the given file descriptor (fd). If you need to convert string data into byte string data, you can use Python’s built-in encode() function.

Example:

import os 

file=os.open('file.txt',flags=os.O_WRONLY)
print(file,os.O_WRONLY)

encodetext=str.encode('hello world')
print(encodetext)

print(os.write(file,encodetext))
os.close(file)

Output:

3 1
b'hello world'
11

The os.read() function lets you read data from a file using its file descriptor. It returns the data in bytes, and you specify how many bytes to read with the length parameter. To get the original data, you can use the decode() function.

Example:

import os 

readfile=os.open('file.txt',flags=os.O_RDONLY)
print(readfile,os.O_RDONLY)

decodetext=os.read(readfile,7).decode()
print(decodetext)

os.close(readfile)

Output:

3 0
hello w

Closing File Descriptors

It’s crucial to close file descriptors after using them to avoid potential issues like running out of available file descriptors. os.close(fd) does just that, freeing up system resources.

Example:

import os 

file=os.open('file.txt',flags=os.O_WRONLY)
print(file,os.O_WRONLY)

os.close(file)

Output:

3 1

OS.fdopen()

The os.fdopen(fd, mode) function serves as an alias for Python’s built-in open() function and returns a file object associated with the provided file descriptor (fd). You can use standard file operations like – read(), write(), and close() on the returned file object.

Example – 1 :

import os 

readwritefile=os.open('file.txt',flags=os.O_RDWR)
fdopenfile_to_write=os.fdopen(readwritefile,mode="w")
print(fdopenfile_to_write)
print(fdopenfile_to_write.write("Hello python, how are you?"))

fdopenfile_to_write.close()

Output :

<_io.TextIOWrapper name=3 mode='w' encoding='cp1252'>
26

Example – 2 :

import os 

readwritefile=os.open('file.txt',flags=os.O_RDWR)
fdopenfile_to_read=os.fdopen(readwritefile,mode="r")
print(fdopenfile_to_read.read(20))
fdopenfile_to_read.close()

fdopenfile_to_write.close()

Output :

Hello python, how ar

Other General Modules