In the previous entry, I introduced Pandas Series. I also compared it with the column of an Excel workbook. Well, following that analogy, DataFrame is the full Excel workbook, where each column is … you guessed it; a Series.

Just like in an Excel workbook, all columns share the same index, and there is a list of columns. Each column will have a distinctive name:

>>> import pandas as pd
>>> idx = range(5)
>>> cols = ['A','B','C']
>>> d = pd.DataFrame(index=idx, columns=cols)
>>> print(d)
     A    B    C
0  NaN  NaN  NaN
1  NaN  NaN  NaN
2  NaN  NaN  NaN
3  NaN  NaN  NaN
4  NaN  NaN  NaN

Since there is no data in the DataFrame, all entries show as NaN (Not a Number). We will fill it with data to show the ways we can address the contents of the DataFrame. For instance, we can access individual elements:

>>> d.loc[0,'A'] = 'Hello'
>>> print(d)
       A    B    C
0  Hello  NaN  NaN
1    NaN  NaN  NaN
2    NaN  NaN  NaN
3    NaN  NaN  NaN
4    NaN  NaN  NaN

There are other methods for doing this, but I prefer to do it always in the same manner, so there are no confusions. That’s why in the entry about Series I recommended to use Series.loc[] to access the elements.
We can also set a full column, or a range of values:

>>> d.loc[:,'B'] = 'Column B' # Sets the value of all the entries in the B column
>>> d.loc[1:4,'A'] = 'A' # Sets the value of the A column from index 1 to 4
>>> print(d)
       A         B    C
0  Hello  Column B  NaN
1      A  Column B  NaN
2      A  Column B  NaN
3      A  Column B  NaN
4      A  Column B  NaN

The : works just as in normal Python slices. Summarizing, if you want to select the whole column, use : as the index range, and if you only want a subset, use [start]:[end] (note that, unlike in slices, [end] is included in the selection).
We can also address a whole row:

>>> d.loc[1,:] = 'NO INFO'
>>> print(d)
         A         B        C
0    Hello  Column B      NaN
1  NO INFO   NO INFO  NO INFO
2        A  Column B      NaN
3        A  Column B      NaN
4        A  Column B      NaN

When we use the DataFrame.loc[] operator, it gives us a handle to a Series or DataFrame (if both the index and column fields were set to a range); so all the operations that were introduced for Series can be executed over the result of the call:

>>> d.loc[:,'A'].apply(len)
0    5
1    7
2    1
3    1
4    1
Name: A, dtype: int64

This example applies the len function to each string of column A. It returns a Series with the length of each entry.
The apply() function can be applied over a DataFrame. Instead of working elementwise, the function is applied over each column or row (depending on the axis parameter):

>>> d.apply(len)
A    5
B    5
C    5
dtype: int64
>>> d.apply(len, axis=1)
0    3
1    3
2    3
3    3
4    3
dtype: int64

In order to apply a function elementwise, another method is used: DataFrame.applymap():

>>> d.applymap(type)
               A              B                C
0      
1        
2      
3      
4      

Similarly to what happened in Series, we can filter columns/rows according to certain conditions. For instance, we can choose the rows that satisfy a certain condition for column A:

>>> d.loc[d.loc[:,'A']!='A',:]
         A         B        C
0    Hello  Column B      NaN
1  NO INFO   NO INFO  NO INFO

Here, d.loc[:,’A’]!=’A’ is used for indexing. The result of this expression on its own would be:

>>> d.loc[d.loc[:,'A']!='A',:]
0     True
1     True
2    False
3    False
4    False
Name: A, dtype: bool

The result is a Series that acts as a mask in the index of d, indicating the rows that match the condition.

In this entry we have seen how to create a DataFrame and access its contents. Of course, there is much much more to say about them, especially in the mathematics field, but I will use it to write future entries in this blog.