Running Bash commands in Python

On my local machine, I run a python script which contains this line This works fine. Then I run the same code on a server and I get the following error message So what I did then is I inserted a print bashCommand which prints me than the command in the terminal before it runs it with os.system(). … Read more

Errno 13 Permission denied Python

Your user don’t have the right permissions to read the file, since you used open() without specifying a mode. Since you’re using Windows, you should read a little more about File and Folder Permissions. Also, if you want to play with your file permissions, you should right-click it, choose Properties and select Security tab. Or if you want to be a little more hardcore, you can … Read more

ValueError: unsupported pickle protocol: 3, python2 pickle can not load the file dumped by python 3 pickle?

You should write the pickled data with a lower protocol number in Python 3. Python 3 introduced a new protocol with the number 3 (and uses it as default), so switch back to a value of 2 which can be read by Python 2. Check the protocolparameter in pickle.dump. Your resulting code will look like this. There is no protocolparameter in pickle.load because pickle can determine … Read more

How to change dataframe column names in pyspark?

There are many ways to do that: Option 1. Using selectExpr. data = sqlContext.createDataFrame([(“Alberto”, 2), (“Dakota”, 2)], [“Name”, “askdaosdka”]) data.show() data.printSchema() # Output #+——-+———-+ #| Name|askdaosdka| #+——-+———-+ #|Alberto| 2| #| Dakota| 2| #+——-+———-+ #root # |– Name: string (nullable = true) # |– askdaosdka: long (nullable = true) df = data.selectExpr(“Name as name”, “askdaosdka as age”) … Read more

SyntaxError invalid token

In Python 3, leading zeros are not allowed on numbers. E.g: Etc. are not allowed, but should be written as 5 and 123 instead. In Python 2, however, the leading zero signifies that the number is an octal number (base eight), so 04 or 03 would mean 4 and 3 in octal, respectively, but 08 would be invalid as it is not a valid octal number. In Python 3, the syntax … Read more

How to append multiple values to a list in Python

You can use the sequence method list.extend to extend the list by multiple values from any kind of iterable, being it another list or any other thing that provides a sequence of values. So you can use list.append() to append a single value, and list.extend() to append multiple values.

Get first row value of a given column

To select the ith row, use iloc: To select the ith value in the Btime column you could use: There is a difference between df_test[‘Btime’].iloc[0] (recommended) and df_test.iloc[0][‘Btime’]: DataFrames store data in column-based blocks (where each block has a single dtype). If you select by column first, a view can be returned (which is quicker than returning a copy) and the original dtype is preserved. In … Read more