Create HIVE Table with multi character delimiter

Question

I want to create a HIVE Table with multi string character as a delimiter such as

CREATE EXTERNAL TABlE tableex(id INT, name STRING) 
ROW FORMAT delimited fields terminated by ','
LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/user/myusername';

I want to have delimiter as a multi string like "~*".

Answer 1

FILELDS TERMINATED BY does not support multi-character delimiters. The easiest way to do this is to use RegexSerDe:

CREATE EXTERNAL TABlE tableex(id INT, name STRING) 
ROW FORMAT 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
  "input.regex" = "^(\\d+)~\\*(.*)$"
)
STORED AS TEXTFILE 
LOCATION '/user/myusername';

Answer 2

Please use MultiDelimitSerde

CREATE EXTERNAL TABlE tableex(id INT, name STRING) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' 
WITH SERDEPROPERTIES ("field.delim"="~*")
STORED AS TEXTFILE
LOCATION '/user/myusername';

Comments

Above solution is working but after setting serde properties, its removing my column comments to 'from deserializer'.

Any idea?

---

Hey, @Atul,

Will you please post what exactly is happening? It will be helpful to understand and resolve your query.