Sqoop split-by problem

Question

I am giving the number of mapper to 10 but still, the sqoop is transferring the data into 5 files only.
sqoop import --connect jdbc:mysql://localhost/retail_db \--username root \--P --table order_items -m 10 \--split-by "order_item_quantity";

When I run the query select distinct(order_item_quantity) from order_item it is giving me the result as 1,2,3,4,5.

Are we telling that the number of the mapper, in this case, is irrelevant?

Answer 1

Hello. The -m or --num-mappers is just a hint to the engine to maintain that degree of parallelism. But it is not mandatory to launch those number of tasks always. The mappers count may vary based on your input data. Sqoop client serializes the data, generates the deserializer and sets the input format and submits the job to be run.