Key Concepts

Review core concepts you need to learn to master this subject

Pandas DataFrames adding columns

Pandas apply() function

Pandas

Pandas DataFrame creation

Selecting Pandas DataFrame rows using logical operators

Pandas DataFrames adding columns

# Specifying each value in the new column: df['newColumn'] = [1, 2, 3, 4] # Setting each row in the new column to the same value: df['newColumn'] = 1 # Creating a new column by doing a # calculation on an existing column: df['newColumn'] = df['oldColumn'] * 5

Pandas DataFrames allow for the addition of columns after the DataFrame has already been created, by using the format df['newColumn'] and setting it equal to the new column’s value.

Pandas apply() function

# Specifying each value in the new column: df['newColumn'] = [1, 2, 3, 4] # Setting each row in the new column to the same value: df['newColumn'] = 1 # Creating a new column by doing a # calculation on an existing column: df['newColumn'] = df['oldColumn'] * 5

The Pandas apply() function can be used to apply a function on every value in a column or row of a DataFrame, and transform that column or row to the resulting values.

By default, it will apply a function to all values of a column. To perform it on a row instead, you can specify the argument axis=1 in the apply() function call.

Pandas

# Specifying each value in the new column: df['newColumn'] = [1, 2, 3, 4] # Setting each row in the new column to the same value: df['newColumn'] = 1 # Creating a new column by doing a # calculation on an existing column: df['newColumn'] = df['oldColumn'] * 5

Pandas is an open source library that is used to analyze data in Python. It takes in data, like a CSV or SQL database, and creates an object with rows and columns called a data frame. Pandas is typically imported with the alias pd.

Pandas DataFrame creation

# Specifying each value in the new column: df['newColumn'] = [1, 2, 3, 4] # Setting each row in the new column to the same value: df['newColumn'] = 1 # Creating a new column by doing a # calculation on an existing column: df['newColumn'] = df['oldColumn'] * 5

The fundamental Pandas object is called a DataFrame. It is a 2-dimensional size-mutable, potentially heterogeneous, tabular data structure.

A DataFrame can be created multiple ways. It can be created by passing in a dictionary or a list of lists to the pd.DataFrame() method, or by reading data from a CSV file.

Selecting Pandas DataFrame rows using logical operators

# Specifying each value in the new column: df['newColumn'] = [1, 2, 3, 4] # Setting each row in the new column to the same value: df['newColumn'] = 1 # Creating a new column by doing a # calculation on an existing column: df['newColumn'] = df['oldColumn'] * 5

In pandas, specific rows can be selected if they satisfy certain conditions using Python’s logical operators. The result is a DataFrame that is a subset of the original DataFrame.

Multiple logical conditions can be combined with OR (using |) and AND (using &), and each condition must be enclosed in parentheses.

Chevron Left Icon
Creating, Loading, and Selecting Data with Pandas
Lesson 1 of 2
Chevron Right Icon
  1. 1
    Pandas is a Python module for working with tabular data (i.e., data in a table with rows and columns). Tabular data has a lot of the same functionality as SQL or Excel, but Pandas adds the power of…
  2. 2
    A DataFrame is an object that stores data as rows and columns. You can think of a DataFrame as a spreadsheet or as a SQL table. You can manually create a DataFrame or fill it with data from a CSV, …
  3. 3
    You can also add data using lists. For example, you can pass in a list of lists, where each one represents a row of data. Use the keyword argument columns to pass a list of column names. df2 …
  4. 4
    We now know how to create our own DataFrame. However, most of the time, we’ll be working with datasets that already exist. One of the most common formats for big datasets is the CSV. *CSV (com…
  5. 5
    When you have data in a CSV, you can load it into a DataFrame in Pandas using .read_csv(): pd.read_csv(‘my-csv-file.csv’) In the example above, the .read_csv() method is called. The CSV file cal…
  6. 6
    When we load a new DataFrame from a CSV, we want to know what it looks like. If it’s a small DataFrame, you can display it by typing print(df). If it’s a larger DataFrame, it’s helpful to be able…
  7. 7
    Now we know how to create and load data. Let’s select parts of those datasets that are interesting or important to our analyses. Suppose you have the DataFrame called customers, which contains the…
  8. 8
    When you have a larger DataFrame, you might want to select just a few columns. For instance, let’s return to a DataFrame of orders from ShoeFly.com: |id|first_name|last_name|email|shoe_type|sh…
  9. 9
    Let’s revisit our orders from ShoeFly.com: |id|first_name|last_name|email|shoe_type|shoe_material|shoe_color| | — | — | — | — | — | — | — | |54791|Rebecca|Lindsay|RebeccaLindsay57…
  10. 10
    You can also select multiple rows from a DataFrame. Here are a few more rows from ShoeFly.com’s orders DataFrame: |id|first_name|last_name|email|shoe_type|shoe_material|shoe_color| |-|-|-|-|-|…
  11. 11
    You can select a subset of a DataFrame by using logical statements: df[df.MyColumnName == desired_column_value] We have a large DataFrame with information about our customers. A few of the many r…
  12. 12
    You can also combine multiple logical statements, as long as each statement is in parentheses. For instance, suppose we wanted to select all rows where the customer’s age was under 30 or the cus…
  13. 13
    Suppose we want to select the rows where the customer’s name is either “Martha Jones”, “Rose Tyler” or “Amy Pond”. |name|address|phone|age| |-|-|-|-| |Martha Jones|123 Main St.|234-567-8910|2…
  14. 14
    When we select a subset of a DataFrame using logic, we end up with non-consecutive indices. This is inelegant and makes it hard to use .iloc(). We can fix this using the method .reset_index(). F…
  15. 15
    You’ve completed the lesson! You’ve just learned the basics of working with a single table in Pandas, including: - Create a table from scratch - Loading data from another file - Selecting certain …
  1. 1
    In the previous lesson, you learned what a DataFrame is and how to select subsets of data from one. In this lesson, you’ll learn how to modify an existing DataFrame. Some of the skills you’ll lea…
  2. 2
    Sometimes, we want to add a column to an existing DataFrame. We might want to add new information or perform a calculation based on the data that we already have. One way that we can add a new co…
  3. 3
    We can also add a new column that is the same for all rows in the DataFrame. Let’s return to our inventory example: |Product ID|Product Description|Cost to Manufacture|Price| |————–|–…
  4. 4
    Finally, you can add a new column by performing a function on the existing columns. Maybe we want to add a column to our inventory table with the amount of sales tax that we need to charge for eac…
  5. 5
    In the previous exercise, we learned how to add columns to a DataFrame. Often, the column that we want to add is related to existing columns, but requires a calculation more complex than multiplic…
  6. 6
    A lambda function is a way of defining a function in a single line of code. Usually, we would assign them to a variable. For example, the following lambda function multiplies a number by 2 and t…
  7. 7
    We can make our lambdas more complex by using a modified form of an if statement. Suppose we want to pay workers time-and-a-half for overtime (any work above 40 hours per week). The following fun…
  8. 8
    In Pandas, we often use lambda functions to perform complex operations on columns. For example, suppose that we want to create a column containing the email provider for each email address in the f…
  9. 9
    We can also operate on multiple columns at once. If we use apply without specifying a single column and add the argument axis=1, the input to our lambda function will be an entire row, not a column…
  10. 10
    When we get our data from other sources, we often want to change the column names. For example, we might want all of the column names to follow variable name rules, so that we can use **df.column_…
  11. 11
    You also can rename individual columns by using the .rename method. Pass a dictionary like the one below to the columns keyword argument: {‘old_column_name1’: ‘new_column_name1’, ‘old_column_name…
  12. 12
    Great job! In this lesson, you learned how to modify an existing DataFrame. Some of the skills you’ve learned include: - Adding columns to a DataFrame - Using lambda functions to calculate comple…

What you'll create

Portfolio projects that showcase your new skills

Pro Logo

How you'll master it

Stress-test your knowledge with quizzes that help commit syntax to memory

Pro Logo