Opening a file and reading the content of a file is one of the common things you would do while doing data analysis.
In this tutorial, we will see 3 examples of reading a text file in Python 3.
One easy way to read a text file and parse each line is to use the python statement “readlines” on a file object.
How To Read all lines in a file at once? Use readlines()
If you want to read all lines of a file at the same time, Python’s readlines() function is for you. Python’s readlines function reads everything in the text file and has them in a list of lines. Here is an example of how to use Python’s readlines.
We first open the file using open() function in read only mode. And use the file handler from opening the file to read all lines using readlines() as follows.
# Open the file with read only permit
f = open('my_text_file.txt', "r")
# use readlines to read all lines in the file
# The variable "lines" is a list containing all lines in the file
lines = f.readlines()
# close the file after reading the lines.
f.close()
We can also read all the lines of a file at once in another way. Basically, we would use the file handler object after opening the file as argument to list() function to get all the lines as a list.
Another way to read lines at once is to simply use
# read all lines at once lines = list(f)
Note that the last character of each line is newline character.
Then you can go over the list of “lines” to parse each line. As you can immediately notice, “readlines” or “list(f) works great for a small text file. However, it is not memory efficient to use if your text files are really big. A better way to read a text file that is memory-friendly is to read the file line by line, that is one line at a time.
Core Python has (at least) two ways to read a text file line by line easily.
How To Read a Text File Line by Line Using While Statement in Python?
Here is the way to read text file one line at a time using “While” statement and python’s readline function. Since we read one line at a time with readline, we can easily handle big files without worrying about memory problems.
# Open the file with read only permit
f = open('my_text_file.txt')
# use readline() to read the first line
line = f.readline()
# use the read line to read further.
# If the file is not empty keep reading one line
# at a time, till the file is empty
while line:
# in python 2+
# print line
# in python 3 print is a builtin function, so
print(line)
# use realine() to read next line
line = f.readline()
f.close()
Another variation of reading a file with while statement and readline statement is as follows. Here the while tests for boolean and read line by line until we reach the end of file and line will be empty.
# file handle fh
fh = open('my_text_file.txt')
while True:
# read line
line = fh.readline()
# in python 2, print line
# in python 3
print(line)
# check if line is not empty
if not line:
break
fh.close()
How To Read a Text File Line by Line Using an Iterator in Python?
One can also use an iterator to read a text file one line at time. Here is how to do it.
fh = open('my_text_file.txt')
for line in fh:
# in python 2
# print line
# in python 3
print(line)
fh.close()
Remembering to close the file handler (“fh”) with the statement “fh.close()” can be difficult initially. One can check if a file handler is closed with
# check if the file file handler is closed or not >fh.closed # true if the file handler is closed True
One can open files in a much simpler way using “with” statement in Python, without having to close the file handler. The with operator creates a context manager and it will automatically close the file for you when you are done with it.
Check here to see how to use “with” statement to open file.
Do you want to read a text file line by line and skip comment lines?, check this post
Do you want to read/load text file numerical data, check this post
Do you have data in a csv or tab limited text file and want to read it in Python? The best option is to use Python’s pandas package. Here is how to load data files in python with Pandas,
