FetchTask directly fetches data, whereas Mapreduce will invoke a map reduce job
<property> <name>hive.fetch.task.conversion</name> <value>minimal</value> <description> Some select queries can be converted to single FETCH task minimizing latency.Currently the query should be single sourced not having any subquery and should not have any aggregations or distincts (which incurrs RS), lateral views and joins. 1. minimal : SELECT STAR, FILTER on partition columns, LIMIT only 2. more : SELECT, FILTER, LIMIT only (+TABLESAMPLE, virtual columns) </description> </property>
Run code snippetExpand snippet
Also there is another parameter hive.fetch.task.conversion.threshold
which by default in 0.10-0.13 is -1 and >0.14 is 1G(1073741824) This indicates that, If table size is greater than 1G use Mapreduce instead of Fetch task
Related Posts:
- Hive: how to show all partitions of a table?
- fs.hdfs.impl.disable.cache caused SparkSQL very slow
- Hive dynamic partitioning
- Difference between hive.tez.container.size and tez.task.resource.memory.mb
- how to write subquery and use “In” Clause in Hive
- get “ERROR: Can’t get master address from ZooKeeper; znode data == null” when using Hbase shell
- Failed to locate the winutils binary in the hadoop binary path
- Name node is in safe mode. Not able to leave
- java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
- First hadoop project error: “Input path does not exist”
- Hive Map-Join configuration mystery
- Difference between `load data inpath ` and `location` in hive?
- What is Keyword Context in Hadoop programming world?
- What’s the best way to search for a UPC code in a Database?
- Optimum way to compare strings in JavaScript?
- What is copy-on-write?
- What is the purpose of the “role” attribute in HTML?
- How can you profile a Python script?
- Cleanest way to copy a constant size array in c++11
- Hadoop “Unable to load native-hadoop library for your platform” warning
- What is the difference between spark.sql.shuffle.partitions and spark.default.parallelism?
- How to efficiently remove duplicates from an array without using Set
- Big O, how do you calculate/approximate it?
- How to Update/Drop a Hive Partition?
- connect to host localhost port 22: Connection refused
- What should be hadoop.tmp.dir ?
- Most efficient way to increment a Map value in Java
- Image rotation algorithm
- Hive’s unix_timestamp and from_unixtime functions
- Difference between INNER JOIN and LEFT SEMI JOIN
- Where is the classpath set for hadoop
- SQL: How to properly check if a record exists
- Fastest way to put contents of Set
to a single String with words separated by a whitespace? - Steps to optimize WordPress in regard to server load and website speed?
- Refactoring WordPress to improve memory performance [closed]
- How to Optimize WP site for millions of posts
- Optimize post insert and delete for bulk operations?
- How to cache json with wp-super cache
- WordPress (MyISAM) database is slow, should I switch to InnoDB?
- How to benchmark a WordPress installation? [closed]
- What is a good alternative to using $content_width for image optimization?
- WP_Query leaking absurd amounts of memory
- Optimizing a Proximity-based Store Location Search on a Shared Web Host?
- What’s a good strategy for planning WordPress DB growth?
- Set Alias for meta_query arguments in get_posts()
- Make Google index the entire post if it is separated into several pages
- How do I optimize a custom post type admin page with 25,000 posts?
- Separate Database Tables For Different Post Types
- Why is home (a lot) slower than other pages?
- How to prevent execution of default query, while preserving ability to use WP_Query in template?
- Long Waiting Times on GoDaddy? [closed]
- Scaling a WP eCommerce site
- Safe to disable wptexturize?
- Running WordPress on low-end hardware/resources?
- What’s the case against transient-ing almost everything that’s mostly static?
- Optimize Multiple Taxonomy Term MySQL Query?
- Where should I host my images?
- Website is slow: advice on optimization
- Preventing YouTube embeds loading multiple instances of player JS?
- Is checking whether the blog is installed necessary?
- How to remove in the wordpress database all posts revisions except the last three?
- Easy way to process search results before displaying
- Speeding Up Bulk Post Creation – wp_insert_post & update_post_meta
- How to properly insert a stylesheet in wp_head
- How to solve ‘Eliminate render-blocking JavaScript and CSS in above-the-fold’ content for multiple files?
- Settings to get the most out of W3 Total Cache
- How to solve slow WordPress site caused by attachment_metadata
- Minimal WordPress load for only `get_option` to work (because ajax…)
- Is there a way to measure server resource (CPU) usage by WP plugins?
- Most efficient way to list all categories and display a post for each of them?
- Defer Code in Widgets – Page Speed
- Speed up WordPress
- If I consider changing my WordPress DB tables to InnoDB, will it have an effect on the way WordPress works?
- Using Multiple Queries of “home_url” vs. Calling a Variable Multiple Times
- Splitting the main query in multiple loops with query_posts and/or pre_get_posts?
- Disable unneeded scripts in a wordpress installation
- Image sizes and order of operations
- Performance-wise, is it better to enqueue a (small) script on every page or test to see if it’s needed?
- WordPress Query is taking more then 20 second and stuck on creating index
- Could a higher number of registered image sizes affect performance?
- Publish a message on facebook after having posted a comment
- How do I edit the htaccess file to optimize my website?
- Continue execution after WP REST API response
- Can’t move jQuery to footer
- Should I remove transients from the wp_options table?
- Optimizing WordPress’s styles and scripts
- What’s faster? One big query, or several smaller ones?
- How to optimize my query filtering out unwanted data?
- Improve wp query perfomance on large db
- Can I prevent “wp_user_level” from getting added to wp_usermeta on registration?
- How to stop wp_postmeta from being called on archive and search pages?
- Very Slow Page – How to Optimize # of Queries?
- WordPress hosting optimized servers – Is this just a sales gimmick? [closed]
- How can I easily reduce the size of WordPress installation from development to production?
- WP Optimization: Removing Orphaned wp_options (especially the autoload ones)
- I’m designing a plugin to create database indexes. Suggestions?
- wordpress with 1.5 million posts
- Using uncss or similar with WordPress?
- WP Responsive images – upload custom image sizes
- How inefficient is it to use a Custom Post for 4-5 sentences & pull all to 1 page?