I ran into a similar problem. It turned out the CSV I had downloaded had no permissions at all. The error message from pandas did not point this out, making it hard to debug. Check that your file have read permissions
From version 0.19.0 of Pandas you can use the lines parameter, like so:
How are you generating your data? See how the output shows that your data is of ‘object’ type? the groupby operations specifically check whether each column is a numeric dtype first. look ↑ Did you initialize an empty DataFrame first and then filled it? If so that’s probably why it changed with the new version … Read more
Use to_datetime, there is no need for a format string the parser is man/woman enough to handle it: To access the date/day/time component use the dt accessor: You can use strings to filter as an example:
You need to select that column: So the syntax here is: You can check the docs and also the 10 minutes to pandas which shows the semantics EDIT If you want to generate a boolean indicator then you can just use the boolean condition to generate a boolean Series and cast the dtype to int this will convert True and False to 1 and 0 respectively:
This is one of the lines in my code where I get the SettingWithCopyWarning: Which I then changed to : This still gives the same warning. How do I get rid of it? Also, I get the same warning for a convert_objects(convert_numeric=True) function that I’ve used, is there any way to avoid that. This is the warning message … Read more
OK, two steps to this – first is to write a function that does the translation you want – I’ve put an example together based on your pseudo-code: You may want to go over this, but it seems to do the trick – notice that the parameter going into the function is considered to be … Read more
First, to convert a Categorical column to its numerical codes, you can do this easier with: dataframe[‘c’].cat.codes.Further, it is possible to select automatically all columns with a certain dtype in a dataframe using select_dtypes. This way, you can apply above operation on multiple and automatically selected columns. First making an example dataframe: Then by using select_dtypes to select the … Read more
A simple pivot might be sufficient for your needs but this is what I did to reproduce your desired output: Just adding a within group counter/index will get you most of the way there but the column labels will not be as you desired: To get closer to your desired output I added the following: … Read more
Here’s an example using apply on the dataframe, which I am calling with axis = 1. Note the difference is that instead of trying to pass two values to the function f, rewrite the function to accept a pandas Series object, and then index the Series to get the values needed. Depending on your use case, it is sometimes … Read more