INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

ETL on SAP BEX report - missing fields

ETL on SAP BEX report - missing fields

(OP)
I've done basic table loads with SSIS and had success but now have a complicated report to load.
The format is similar to this:

CODE -->

TransactionID UserID  firstname lastname itemID itemdescription itemcost
100001        1001    Joe       Jones     abc    black shoes    $50.
100002        1002    Jane      Johnson   bcd    red shoes      $60.
                                          cde    green shoes    $61.
                                          def    blue shoes     $60.
100003        1003    Jay       Jackson   efg    orange shoes   $50. 

The problem is to impute or fill in fields missing from the report, to fill in all fields
during the ETL. This would be a simple programming task in Perl or Python, but how
do you approach this in SSIS?

Any pointers or recommendations greatly appreciated!

RE: ETL on SAP BEX report - missing fields

Where does this missing data come from?

----------------------------------------
www.fountain.me.uk

Be who you are and say what you feel, because those who mind don't matter and those who matter don't mind - Bernard Baruch

Computer Science is no more about computers than astronomy is about telescopes - EW Dijkstra
----------------------------------------

RE: ETL on SAP BEX report - missing fields

I'm assuming the missing data is from the previous line on the report that has those filled.

Pure SSIS - don't advise it if volumes are big.

Load the file to a recordset object

On a dataflow add script as source, and on this script you read the recordset above in a loop, storing the values of each row and sending the values of the previous row to the output collection if they are empty

mix of SSIS/T-SQL.

Load the file onto a staging table adding a row number while doing it (either identity column or a row generator on SSIS)
Then using t-sql perform a update of the rows with the required

Regards

Frederico Fonseca
SysSoft Integrated Ltd
www.syssoft-int.com

FAQ219-2884: How Do I Get Great Answers To my Tek-Tips Questions?
FAQ181-2886: How can I maximize my chances of getting an answer?

RE: ETL on SAP BEX report - missing fields

(OP)
Frederico,

Thanks, yes missing fields come from a previous record. OK, I will look at recordset object + T-SQL.

Red Flag This Post

Please let us know here why this post is inappropriate. Reasons such as off-topic, duplicates, flames, illegal, vulgar, or students posting their homework.

Red Flag Submitted

Thank you for helping keep Tek-Tips Forums free from inappropriate posts.
The Tek-Tips staff will check this out and take appropriate action.

Reply To This Thread

Posting in the Tek-Tips forums is a member-only feature.

Click Here to join Tek-Tips and talk with other members!

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close