File read write

Reading data from file

Being able to read data from files and write the data generated by your program is an essential part of programming.

Here we will open a file named xps-data.txt which is stored in the datafiles directory.

Our data file contains two columns of numbers; energy and intensity. We can read the data following way:

# create two empty lists to store our energy and intensity values
energy = []
intensity = []

fid = open('../datafiles/xps-data.txt', 'r')
# read one line at a time
data = fid.readlines()
fid.close()

# number of lines we have
lines = len(data);
for lines in range(lines):
    data_row = data[lines]

    # remove the last newline character from each line
    data_row = data_row[:-1]

    # split in the tab character to separate energy and intensity strings
    data_row = data_row.split('\t')

    # store them in our energy and intensity variables as float
    energy.append(float(data_row[0]))
    intensity.append(float(data_row[1]))

Now that our data is stored in the energy and intensity variables. Best way to visualize our data is to make a plot using matplotlib:

import matplotlib.pyplot as plt
%matplotlib inline

plt.figure(figsize = (10, 8))
plt.plot(energy, intensity)
plt.xlabel('Kinetic energy (eV)')
plt.ylabel('Intensity (a.u.)')
plt.show()

Writing data to file

Now that we are able to read data from a file, and use in our program, let us try to generate and save some data.

import numpy as np
x = np.linspace(-2*np.pi, 2*np.pi, num = 100)
y = (np.sin(x)/x)**2

plt.figure(figsize = (10, 8))
plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('f(x)')
plt.show()

We can save the data as follows:

fid = open('../datafiles/data.txt', 'w')

for index in range(len(x)):
    fid.write('{0}\t{1}\n'.format(x[index], y[index]))
fid.close()

We have stored our data in a file named data.txt.

info

The read mode is the default while opening files, if you need only read permission while opening a file, you can simply omit file mode argument after the file name. It is important to close the files when it is not required in order to clear system resources. See the with statement below, which automatically closes the files.

Using with open

We can use another method to handle file operations.

with open('data.txt', 'r') as f:
    data = f.read()

There is an advantage of using the with statement: the file is automatically closed when it’s not needed anymore.

File I/O using numpy

numpy has a helpful library to read/write formatted data files. We can do above tasks following way using numpy:

# read from file
data = np.loadtxt("../datafiles/xps-data.txt")

# plot
plt.plot(data[:, 0], data[:, 1])
plt.xlabel('Kinetic energy (eV)')
plt.ylabel('Intensity (a.u.)')
plt.show()

# save to file
data = np.array([x, y]).T
np.savetxt("../datafiles/data.txt", data)

For loading a bit more complex data file with headers and comments lines, see this example: Quantum Design SQUID data loader notebook.