How to filter rows containing a string in several columns

Filter Rows Containing a String in Several Columns

Suppose you have a Pandas DataFrame with multiple columns, and you need to filter the rows that contain a specific string in any of those columns. Here's a step-by-step guide to achieve this:

Step 1: Gather the required columns

columns_to_filter = ['column1', 'column2', 'column3']

Replace column1, column2, and column3 with the actual column names you want to filter.

Step 2: Create a condition using apply() and str.startswith()

cond = df[columns_to_filter].apply(lambda col: col.str.startswith('d')).all(axis=1)

In this code, we use apply() to apply the str.startswith() function to each column in columns_to_filter. The str.startswith() function checks if a string starts with a specific substring. In this example, we're checking if any of the values in the selected columns start with the letter "d".

Step 3: Filter the DataFrame using the condition

filtered_df = df[cond]

The filtered_df variable now contains only the rows where at least one value in the specified columns starts with "d".

Additional Considerations:

• Multiple Strings: You can also check if each column starts with a different string. For example:

lookup = {
    "text": "a",
    "line": "b",
    "note": "c"
}

cond = df[[col_name]].apply(lambda col: col.str.startswith(lookup[col.name])).all(axis=1)

• Continuous Columns: If you have continuous columns (e.g., numerical data) instead of strings, you can use a different condition. For example:

cond = (df.loc[:, 1:] > 0).all(axis=1)

This code checks if all values in columns starting from the second column (df.loc[:, 1:]) are greater than 0.