hive-configuration - Read For Learn

Hive Map-Join configuration mystery

These parameters are used to make decision on when to use Map Join against Common join in hive, which ultimately affects query performance at the end. Map join is used when one of the join tables is small enough to fit in the memory, so it is very fast. here’s the explanation of all parameters: hive.auto.convert.join When this parameter set … Read more

Why is Fetch task in Hive works faster than Map-only task?

FetchTask directly fetches data, whereas Mapreduce will invoke a map reduce job Run code snippetExpand snippet Also there is another parameter hive.fetch.task.conversion.threshold which by default in 0.10-0.13 is -1 and >0.14 is 1G(1073741824) This indicates that, If table size is greater than 1G use Mapreduce instead of Fetch task more detail