How to improve paralelized computing in AWS EC2?

I am using mpi4py and MPICH (installed with conda) to parallelize the training of a reinforcement learning system across several CPUs ( using an AWS EC2 instance, namely, a c5.x12) with Ubuntu. I have benchmarked the performance and the amount of training per unit of time increases 30% (when 5 processes are used) with respect to the training with one single process. However, when I use 5 processes in my local computer I get an increase in the amount of training per unit of time of 300% with respect to the training with one single process.

In my computer I use Windows and Microsoft MPI, which I think is based on MPICH, so what can cause this performance difference? How can I get the best from AWS?

Related Posts:

  1. AWS Lambda:The provided execution role does not have permissions to call DescribeNetworkInterfaces on EC2
  2. AWS – Disconnected : No supported authentication methods available (server sent :publickey)
  3. What is difference between Lightsail and EC2?
  4. What is the difference between Amazon SNS and Amazon SQS?
  5. AWS: What does 0.0.0.0/0 and ::/0 mean?
  6. What is the difference between Elastic Beanstalk and CloudFormation for a .NET project?
  7. How can I resolve the error “The security token included in the request is invalid” when running aws iam upload-server-certificate?
  8. DynamoDB – Key element does not match the schema
  9. AWS CLI S3 A client error (403) occurred when calling the HeadObject operation: Forbidden
  10. 403 ERROR The request could not be satisfied
  11. How can I get the size of an Amazon S3 bucket?
  12. AWS RDS connection limits
  13. What is difference between Application Buffer and System Buffer
  14. What is the meaning of the word logits in TensorFlow?
  15. program ended prematurely and may have crashed. exit code 0xc0000005
  16. Expected 2D array, got 1D array instead error
  17. How can I one hot encode in Python?
  18. Error in Python script “Expected 2D array, got 1D array instead:”?
  19. Error in Python script “Expected 2D array, got 1D array instead:”?
  20. What is the use of train_on_batch() in keras?
  21. What’s the difference between torch.stack() and torch.cat() functions?
  22. Cross Entropy in PyTorch
  23. What is cross-entropy?
  24. DynamoDB : The provided key element does not match the schema
  25. word2vec: negative sampling (in layman term)?
  26. What is the difference between sparse_categorical_crossentropy and categorical_crossentropy?
  27. boto3 client NoRegionError: You must specify a region error only sometimes
  28. How to implement the Softmax function in Python
  29. Backward function in PyTorch
  30. What is the difference between np.mean and tf.reduce_mean?
  31. Intuition for perceptron weight update rule
  32. What is the meaning of ‘for _ in range()
  33. ValueError: x and y must be the same size
  34. How can I use wildcards to `cp` a group of files with the AWS CLI
  35. AWS lambda function error – Unable to import module ‘index’: Error
  36. What is the difference between Amazon ECS and Amazon EC2?
  37. Access denied; you need (at least one of) the SUPER privilege(s) for this operation
  38. Difference between Amazon EC2 and AWS Elastic Beanstalk
  39. Access denied; you need (at least one of) the SUPER privilege(s) for this operation
  40. What does `Fatal Python error: PyThreadState_Get: no current thread` mean?
  41. What’s the difference between scikit-learn and tensorflow? Is it possible to use them together?
  42. How to initialize weights in PyTorch?
  43. Calculate the Cumulative Distribution Function (CDF) in Python
  44. Cross Validation in Keras
  45. “invalid ELF header” when using the nodejs “ref” module on AWS Lambda
  46. Scikit-learn GridSearch giving “ValueError: multiclass format is not supported” error
  47. Laravel 5 Class ‘Collective\Html\HtmlServiceProvider’ not found on AWS
  48. Error “You must specify a region” when running any aws CLI command
  49. Google app engine or amazon web services
  50. S3 Bucket action doesn’t apply to any resources
  51. “There was a problem with the requested skill’s response” on Alexa Simulator
  52. How to implement the ReLU function in Numpy
  53. Difference between AWS DynamoDB vs. AWS DocumentDB(Newly launched service)? [closed]
  54. AWS – Create a record set for an s3 static website
  55. ValueError: multiclass format is not supported
  56. Naive Bayes vs. SVM for classifying text data
  57. ImportError(‘Could not import PIL.Image. ‘ working with keras-ternsorflow
  58. Nginx error: client intended to send too large body
  59. Is there an Amazon.com API to retrieve product reviews?
  60. SQS vs RabbitMQ
  61. RuntimeError: dimension out of range (expected to be in range of [-1, 0], but got 1)
  62. AWS s3 api error: specified bucket does not exist
  63. Your WSGIPath refers to a file that does not exist
  64. Amazon Workspaces VM prevented from logging in to WordPress on DreamHost
  65. How can I upgrade to Java 1.8 on an Amazon Linux Server?
  66. Amazon Cloudfront with S3. Access Denied
  67. What Linux distribution is the Amazon Linux AMI based on?

Leave a Comment