Looking for ETL Recommendations

Looking for ETL Software recommendations. Epicor experience preferred, but not required.

I am in the middle of using Talend Open Studio for a data integration project (https://www.talend.com/products/talend-open-studio/).

I’m liking it so far, the learning curve wasn’t too bad and there is plenty of documentation help. When you build the project it creates a java jar file to schedule and execute your project and the entire thing uses java/eclipse so its pretty extensible.

I am taking a pretty poorly formatted text file full of measurement data and parsing it, attaching the epicor partnum and lotnum then inserting it to MSSQL.

If cost is a concern, Open Studio is free.

1 Like

Thanks Jason,
I am playing with Talend too. Looks pretty good but the Interface (Eclipse) has been a nightmare and I can’t find many docs for it. I’l poke around some more though.
I’m specifically looking to consume web services and REST calls and I’ve had little to no luck with it yet. But I’ve only spent an hour or two playing with it.

I’ve been playing with Scibe too (very nice)… but VERY VERY expensive

Thanks!

I used scribe at my last job, both cloud and on-prem. It was very nice, we used it to sync our Vantage 8 DB with a cloud based MS Dynamics CRM instance. The support was great and we never had issues with it. It does cost a fortune though.

Talend has been my life for the last 2 weeks. Let me know if have any questions, I haven’t used the tREST component though.

Eclipse isn’t too bad as long as you take into account it’s free. I did a little android and Linux dev on it - but after VS, it certainly leaves room for improvement.

I’ve been using the Pentaho Open Source BI Suite Community Edition (CE), now owned by Hitachi (https://community.hds.com/docs/DOC-1009931). The ETL portion (aka Kettle or PDI) can be stand-alone. I use the free community version. It’s pretty versatile and I haven’t found anything I can’t do with the free version. I have used it to automate data extraction from web services via REST, databases, CSV files, and spreadsheet files attached to emails.

It’s java based and has command line tools that can be used for task scheduler (windows) or cron (linux).

1 Like

Thanks Marvin
I’ll check it out

-Jose

I like SSIS(SSDT) and MS Analysis stack which is included in 2016 standard
license. We have been using this combined with PowerBI to drive some really
nice self service BI combined with some rolled out reports as well. You can
even leverage SSRS for distrobution.

[image: cid:image003.jpg@01D15793.639F5350] TACONY CORPORATION

*Nathan Woolen *| ERP Specialist

Email: n.woolen@tacony.com | Web: www.tacony.com

Office: 636.349.3000 x 1228

This electronic message, including any attachments, may contain
proprietary, confidential or privileged information for the sole use of the
intended recipient(s).

You are hereby notified that any unauthorized disclosure, copying,
distribution, or use of this message is prohibited. If you have received
this message in error, please immediately notify the sender by reply e-mail
and delete it.

Thanks Nathan, yeah I’ve used SSIS before nice package just looking for something a bit more advanced. Though I haven’t looked at the new features in 2016

Is Talend appropriate for transforming data when migrating from legacy erp to Epicor? I am writing SQL scripts to extract data from the legacy erp (sql server) including CASE statements for 1 to 1 data conversions. Now I’m trying to determine the best way of storing conversion rules such as new Part.SearchWord values when the old system allows 10 characters for the short description but Epicor only allows 8 characters for the SearchWord field. I don’t want to update the old system, so I’m thinking a conversion table with Part and NewSearchWord needs to be set up and used to transform the data after running the SQL extract and before loading into Epicor DMT.

Thanks for any insights based on your experiences.
Regards, Randall

DMT may be a better choice for a one time load

-Jose

Thanks Jose… we will be using DMT for the load into Epicor. What I’m wondering is how people have handled the data transformation when data needs to be changed after it is extracted from the legacy system and before it is loaded into Epicor (via DMT).

Master data will change in the legacy system right up until final migration, so I don’t like the idea of manually preparing data records outside of the legacy ERP.

Talends performance left a bit to be desired. What about using SSIS assuming you are coming from a SQL db

-Jose

We have looked at a few and pretty much chosen Power BI. Cost wise its super cheap with $17 per month per editing seat, users viewing is free. If you want to have on premise then its very costly but if your ok with the cloud then its a great option…if your using office 365 its essentially stored in the same location.

Thanks for both suggestions, Jose and Paul.

Randall - At my last company’s go-live, we did our transformations as Access queries against the legacy db, then exported them as .CSV’s to be imported by DMT…

1 Like

We did something similar, created a database that contained multiple stored procedures and views and combined all the data. We produced out DMT Template output from there.

We found the SP’s the easiest, but I am sure it has been mentioned before, it’s really about the skills and experience you have at your disposal will impact on the decison on what way to go.

Dare I say having a good understanding and data dictionary of your source schema goes a long way.