SQOOP export want to move million records to netezza from hive guidance required

Question

Hi,

We were trying to export the data using below command, the speed at which it gets records is 2000 records every 15 minutes

Kindly do provide us few parameters which speeds up the process

sqoop --connect "jdbc:netezza://server/db?schema=schema" --table nz_table --username nz_usr -P --hcatalog-database hdw_db --hcatalog-table hdp_tbl --num-mappers 20 ;

Thanks

Subramanian

Answer 1

Hi@Subramanian,

You can use the following parameter in your code.

The property sqoop.export.records.per.statement to specify the number of records that will be used in each insert statement.

Also, you can set how many rows will be inserted per transaction with the sqoop.export.statements.per.transaction property.

Use this parameters and try once.