Using Java Classes in Talend

Question

I have created 3 classes namely PageScanner, UnitArray and Main. Then I created routines of PageScanner and UnitArray classes and using tJava component I am calling the Main class and the methods from these two classes. But I am not able to fetch the methods.

I also tried loading the routines but got the following error: Exception in thread "main" java.lang.NoSuchFieldError: INSTANCE

Following are my three classes:

1. PageScanner.class

package page_scraper;

import com.gargoylesoftware.htmlunit.Page;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.WebClientOptions;
import com.gargoylesoftware.htmlunit.html.FrameWindow;
import com.gargoylesoftware.htmlunit.html.HtmlButtonInput;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlOption;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import com.gargoylesoftware.htmlunit.html.HtmlSelect;
import com.gargoylesoftware.htmlunit.html.HtmlTextInput;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintStream;
import java.io.Writer;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
import page_scraper.UnitArray;

public class PageScraper {
    public void Scrape() throws IOException {
        try {
            UnitArray object = new UnitArray();
            ArrayList<String> unitList = object.getUnitArray();
            WebClient webClient = new WebClient();
            webClient.getOptions().setThrowExceptionOnScriptError(false);
            webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
            HtmlPage page = (HtmlPage)webClient.getPage("");
            List frames = page.getFrames();
            HtmlPage page1 = (HtmlPage)((FrameWindow)frames.get(0)).getEnclosedPage();
            HtmlTextInput settlementDay = (HtmlTextInput)page1.getHtmlElementById("param5");
            HtmlSelect period = (HtmlSelect)page1.getHtmlElementById("param6");
            HtmlOption periodOption = period.getOption(1);
            HtmlTextInput unitId = (HtmlTextInput)page1.getHtmlElementById("param1");
            HtmlButtonInput button = (HtmlButtonInput)page1.getHtmlElementById("go_button");
            String outputLocation = String.valueOf(System.getProperty("user.home")) + "/Documents/output.csv";
            FileWriter fileWriter = new FileWriter(outputLocation);
            String errorLocation = String.valueOf(System.getProperty("user.home")) + "/Documents/error.csv";
            FileWriter errorWriter = new FileWriter(errorLocation);
            int i = 0;
            while (i < unitList.size()) {
                int x = 0;
                while (x < 365) {
                    String errorData;
                    SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
                    Calendar cal = Calendar.getInstance();
                    cal.add(5, - x);
                    String dateValue = dateFormat.format(cal.getTime());
                    System.out.println(dateValue);
                    settlementDay.setValueAttribute(dateValue);
                    period.setSelectedAttribute(periodOption, true);
                    unitId.setValueAttribute(unitList.get(i));
                    System.out.println(unitList.get(i));
                    try {
                        button.click();
                        HtmlPage page2 = (HtmlPage)((FrameWindow)frames.get(1)).getEnclosedPage();
                        String pageSource = page2.asXml();
                        int firstIndex = pageSource.indexOf("csv=") + 38;
                        int secondIndex = pageSource.indexOf("n\"") + 1;
                        String csvData = pageSource.substring(firstIndex, secondIndex);
                        fileWriter.append(csvData);
                    }
                    catch (ClassCastException e) {
                        errorData = String.valueOf(dateValue) + " " + unitList.get(i) + System.getProperty("line.separator");
                        System.out.println(errorData);
                        errorWriter.append(errorData);
                        continue;
                    }
                    catch (StringIndexOutOfBoundsException e) {
                        errorData = String.valueOf(dateValue) + " " + unitList.get(i) + System.getProperty("line.separator");
                        System.out.println(errorData);
                        errorWriter.append(errorData);
                        continue;
                    }
                    ++x;
                }
                ++i;
            }
            webClient.close();
            fileWriter.close();
            errorWriter.close();
        }
        catch (IOException e) {
            e.printStackTrace();
        }
    }
}

2. UnitArray.class

package page_scraper;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;

public class UnitArray {
    public ArrayList<String> getUnitArray() {
        String csvList = "abc,xyz";
        ArrayList<String> list = new ArrayList<String>(Arrays.asList(csvList.split(",")));
        return list;
    }
}

3. Main.class

package page_scraper;

import page_scraper.PageScraper;

public class main {
    public static void main(String[] args) throws Exception {
        PageScraper test = new PageScraper();
        test.Scrape();
    }
}

Can anyone suggest how can I call these classes and the methods as well.

Answer 1

While working with routines, the very 1st and most important thing is make sure your routines are inside your routine package.

package routines;
public class PageScraper {
   public void Scrape() {
       System.out.println("PageScraper.Scrape");
   }
}

Once you ensure that, you can just drag and drop your routines into the workspace.

Then you can use your class as following:

Make sure to change the import page_scraper.UnitArray into import routines.UnitArray in PageScraper class.

Hope this helps.

To learn more about Talend' it's recommended to join Talend training today.

Thanks.