The csv file might contain very huge fields, therefore increase the field_size_limit
:
import sys import csv csv.field_size_limit(sys.maxsize)
sys.maxsize
works for Python 2.x and 3.x. sys.maxint
would only work with Python 2.x (SO: what-is-sys-maxint-in-python-3)
Update
As Geoff pointed out, the code above might result in the following error: OverflowError: Python int too large to convert to C long
. To circumvent this, you could use the following quick and dirty code (which should work on every system with Python 2 and Python 3):
import sys import csv maxInt = sys.maxsize while True: # decrease the maxInt value by factor 10 # as long as the OverflowError occurs. try: csv.field_size_limit(maxInt) break except OverflowError: maxInt = int(maxInt/10)
Related Posts:
- TypeError: list indices must be integers or slices, not str
- Writing a pandas DataFrame to CSV file
- Writing a pandas DataFrame to CSV file
- IndexError: too many indices for array
- ValueError : I/O operation on closed file
- Pandas: ValueError: cannot convert float NaN to integer
- csv.Error: iterator should return strings, not bytes
- Python – Reading and writing csv files with utf-8 encoding
- Python Error io.UnsupportedOperation: not readable
- load csv into 2D matrix with numpy for plotting
- Dump a NumPy array into a csv file
- (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape
- How to load a tsv file into a Pandas DataFrame?
- Difference between writerow() and writerows() methods of Python csv module
- Create a .csv file with values from a Python list
- Convert XML to CSV file
- TypeError: list indices must be integers or slices, not str
- Convert from CSV to array in Python
- ValueError: cannot index with vector containing NA / NaN values
- (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape
- Creating a dictionary from a csv file?
- Python import csv to list
- ValueError: x and y must be the same size
- OSError: Initializing from file failed on csv in Pandas
- append new row to old csv file python
- Writing a dictionary to a csv file with one line for every ‘key: value’
- Python CSV error: line contains NULL byte
- Python CSV Error: sequence expected
- AttributeError: ‘float’ object has no attribute ‘split’4
- How to add pandas data to an existing csv file?
- convert csv file to list of dictionaries
- CSV new-line character seen in unquoted field error
- Writing Python lists to columns in csv
- Error in Reading a csv file in pandas[CParserError: Error tokenizing data. C error: Buffer overflow caught – possible malformed input file.]
- How to read a CSV file from a URL with Python?
- How to import a csv-file into a data array?
- bash: pip: command not found
- Function for Factorial in Python
- TypeError: a bytes-like object is required, not ‘str’ when writing to a file in Python3
- Purpose of “%matplotlib inline”
- How do I update Anaconda?
- TypeError: ‘str’ object is not callable (Python)
- Dijkstra’s algorithm in python
- How to create a GUID/UUID in Python
- Can’t fix “zipimport.ZipImportError: can’t decompress data; zlib not available” when I type in “python3.6 get-pip.py”
- Arrays used as indices must be of integer (or boolean) type
- Are static class variables possible in Python?
- Why am I getting “LinAlgError: Singular matrix” from grangercausalitytests?
- ‘method’ object is not subscriptable. Don’t know what’s wrong
- How do I copy a file in Python?
- How do you read from stdin?
- Union of two lists in Python
- Type error: cannot convert the series to
- Reading a binary file with python
- How to determine a Python variable’s type?
- Could not find a version that satisfies the requirement
- Removing Conda environment
- Cannot convert list to array: ValueError: only one element tensors can be converted to Python scalars
- Start with pyglet or pygame?
- Django 2.1.3 Error: __init__() takes 1 positional argument but 2 were given
- Python error load JSON code of google API
- How to convert IPython notebooks to PDF and HTML?
- ‘Jupyter’ is not recognized as an internal or external command
- How to print spaces in Python?
- installing urllib in Python3.6
- What is sys.maxint in Python 3?
- How do you get the magnitude of a vector in Numpy?
- Finding all possible permutations of a given string in python
- pandas create new column based on values from other columns / apply a function of multiple columns, row-wise
- Parsing HTML using Python
- How can I install the latest Anaconda with wget
- sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 74 supplied
- TypeError: ‘int’ object does not support item assignment
- Convert datetime to Unix timestamp and convert it back in python
- Defining lists as global variables in Python
- How to install PyGame on Python 3.4?
- ImportError: No module named ‘cv2’ Python3
- bbox_to_anchor and loc in matplotlib
- Python: Can a function return an array and a variable?
- Python 101: Can’t open file: No such file or directory
- super() fails with error: TypeError “argument 1 must be type, not classobj” when parent does not inherit from object
- Appending values to dictionary in Python
- TypeError: argument of type ‘NoneType’ is not iterable
- How to prettyprint a JSON file?
- How to load all modules in a folder?
- built-in range or numpy.arange: which is more efficient?
- How do I rotate an image around its center using Pygame?
- How can I copy a Python string?
- How to completely uninstall python 2.7.13 on Ubuntu 16.04
- Calculate weighted average using a pandas/dataframe
- Telling Python to save a .txt file to a certain directory on Windows and Mac
- Spell Checker for Python
- TypeError: coercing to Unicode: need string or buffer, int found
- How to fix SSL issue SSL_CTX_use_certificate : ca md too weak on Python Zeep
- Paramiko’s SSHClient with SFTP
- Package libffi was not found in the pkg-config search path REDHAT6.5
- Line is too long. Django PEP8
- TypeError: zip argument #2 must support iteration
- CS231n: How to calculate gradient for Softmax loss function?
- How to create a new database using SQLAlchemy?