Pyspark: Exception: Java gateway process exited before sending the driver its port number

One possible reason is JAVA_HOME is not set because java is not installed.

I encountered the same issue. It says

Exception in thread "main" java.lang.UnsupportedClassVersionError: org/apache/spark/launcher/Main : Unsupported major.minor version 51.0
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:643)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
    at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:296)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:406)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/spark/python/pyspark/conf.py", line 104, in __init__
    SparkContext._ensure_initialized()
  File "/opt/spark/python/pyspark/context.py", line 243, in _ensure_initialized
    SparkContext._gateway = gateway or launch_gateway()
  File "/opt/spark/python/pyspark/java_gateway.py", line 94, in launch_gateway
    raise Exception("Java gateway process exited before sending the driver its port number")
Exception: Java gateway process exited before sending the driver its port number

at sc = pyspark.SparkConf(). I solved it by running

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

which is from https://www.digitalocean.com/community/tutorials/how-to-install-java-with-apt-get-on-ubuntu-16-04

Related Posts:

  1. Spark RDD to DataFrame python
  2. How to change dataframe column names in pyspark?
  3. How to change dataframe column names in pyspark?
  4. How to join on multiple columns in Pyspark?
  5. environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON
  6. bash: pip: command not found
  7. ImportError: No module named pip
  8. ImportError: No module named pip
  9. How do I install pip on macOS or OS X?
  10. ImportError: No module named pandas. Pandas installed pip
  11. Could not install packages due to a “Environment error :[error 13]: permission denied : ‘usr/local/bin/f2py'”
  12. Could not install packages due to an EnvironmentError:
  13. import opencv vs import cv2
  14. socket.error: [Errno 48] Address already in use
  15. socket.error: [Errno 48] Address already in use
  16. Why is the apt-get function not working in the terminal on Mac OS X v10.9 (Mavericks)?
  17. Where do I find the bashrc file on Mac?
  18. bash: pip: command not found
  19. ModuleNotFoundError: No module named ‘matplotlib’
  20. from PIL import Image – ImportError: No module named PIL
  21. Virtualenv Command Not Found
  22. How to join on multiple columns in Pyspark?
  23. tar: Unrecognized archive format error when trying to unpack flower_photos.tgz, TF tutorials on OSX
  24. Segmentation fault: 11 in OS X
  25. Could not install packages due to an EnvironmentError: [Errno 13]
  26. Converting Pandas dataframe into Spark dataframe error
  27. Tkinter error: Couldn’t recognize data in image file
  28. How can I install the Beautiful Soup module on the Mac?
  29. How do I install pip on macOS or OS X?
  30. Homebrew brew doctor warning about /Library/Frameworks/Python.framework, even with brew’s Python installed
  31. While else statement equivalent for Java?
  32. How do I lowercase a string in Python?
  33. Difference between del, remove, and pop on lists
  34. How to use the pass statement
  35. How do you round UP a number?
  36. How do I update\upgrade pip itself from inside my virtual environment?
  37. ImportError: DLL load failed: The specified module could not be found
  38. Replacements for switch statement in Python?
  39. Correct way to write line to file?
  40. How to condense if/else into one line in Python?
  41. Depth-first search (DFS) code in python
  42. How to use Python to execute a cURL command?
  43. What does %s mean in a python format string?
  44. Redirecting to URL in Flask
  45. TypeError: ‘float’ object is not callable
  46. Plotting a 2D heatmap with Matplotlib
  47. Purpose of __repr__ method?
  48. json.dumps vs flask.jsonify
  49. How to print like printf in Python3?
  50. “Defaulting to user installation because normal site-packages is not writeable” python message
  51. How to install pytorch in Anaconda with conda or pip?
  52. Python – Reading and writing csv files with utf-8 encoding
  53. Python’s equivalent of && (logical-and) in an if-statement
  54. Replace values in list using Python
  55. Trouble updating to Anaconda Navigator 1.10.0 (MacOS)
  56. Binary numbers in Python
  57. Django: ImproperlyConfigured: The SECRET_KEY setting must not be empty
  58. builtins.TypeError: must be str, not bytes
  59. How do you read from stdin?
  60. takes 1 positional argument but 2 were given
  61. What is dtype(‘O’), in pandas?
  62. Call a function from another file?
  63. What’s the difference between %s and %d in Python string formatting?
  64. What is the difference between Jupyter Notebook and JupyterLab?
  65. Python 3 ImportError: No module named ‘ConfigParser’
  66. Group by index + column in pandas
  67. Why am I getting a FileNotFoundError?
  68. Anaconda Navigator won’t launch (windows 10)
  69. What are all possible pos tags of NLTK?
  70. Can’t open Jupyter notebook with Anaconda
  71. Pickle , read in data , unsupported pickle protocol: 3 python 2.7
  72. pygame clock.tick() vs framerate in game main loop
  73. Is there a built-in function to print all the current properties and values of an object?
  74. How to troubleshoot an “AttributeError: __exit__” in multiproccesing in Python?
  75. TypeError : Unhashable type
  76. Read PDF in Python and convert to text in PDF
  77. Importing variables from another file in Python
  78. How to read a file without newlines?
  79. Encrypt & Decrypt using PyCrypto AES 256
  80. Error: Segmentation fault (core dumped)
  81. can’t open tensorboard 0.0.0.0:6006 or localhost:6006
  82. How to add a new row to an empty numpy array
  83. Logical indexing with lists
  84. How to set timeout on python’s socket recv method?
  85. How does str(list) work?
  86. Automatic indentation for Python in Notepad++
  87. pandas groupby sort within groups
  88. Turn a single number into single digits Python
  89. Is there a head and tail method for Numpy array?
  90. What should I use to open a url instead of urlopen in urllib3
  91. How to find out whether a file is at its `eof`?
  92. TypeError: Image data can not convert to float
  93. coercing to Unicode: need string or buffer, NoneType found when rendering in django admin
  94. Does python support character type?
  95. Styling multi-line conditions in ‘if’ statements?
  96. What is related_name used for?
  97. Symbol not found: __PyCodecInfo_GetIncrementalDecoder
  98. How can I tail a log file in Python?
  99. How to write unicode strings into a file?
  100. python-How to set global variables in Flask?

Leave a Comment