Getting Started with ASP.NET MVC - Part 5: How to do programming with razor syntax
First Time? You can support us by signing up. It takes only 5 seconds. Click here to sign up. If you already have an account, click here to login.


Upload Image Close it
Select File

SSIS scenarios...
Browse by Tags · View All
MSBI 27
SSIS 24
BRH 15
#BI 14
SQL Server 12
Script Component 9
#SQLSERVER 8
#SQL Server 6
Script Component Source 5
For EAch loop 5

Archive · View All
August 2010 4
March 2011 3
September 2010 3
June 2011 2
November 2010 2
May 2010 2
November 2009 2
March 2010 2
April 2012 1
September 2011 1

SSIS - Dynamically set Flat File Connection Manager

Mar 14 2010 12:09PM by Sudeep Raj   

Hi,

In the SSIS forum very often there are questions on dynamic name for a flat file connection manager

I have split it in two parts:

  • A single file name needs to be set up dynamically
  • Multiple file names need to be set up dynamically per execution.

In this post I will talk about the first scenario.


Scenario:

For simplicity sake I will be using a case where the file name is saved in a DB table.

Need to fetch the file name from a database table.

Need to use a Data Flow Task which has a Flat File Source.

This flat file source needs the file name to be set up dynamically at run time.

The folder for the file is fixed.





Create a string Variable: FileName with the scope as the package(Practice in my case). Set the value of the variable to a file name that exists (I set it to Demo.txt) just while development.




Now set up the Flat File Connection Manager for the above file that exists and has the same meta data as the file that will be used later.


File name: Click on the Browse button and select the file.

I am using the other default settings.





Click on the Columns tab to verify the columns.

Then click OK.




To set up the Execute SQL task to fetch the file name and save it to a user variable FileName.

In the Execute SQL task Editor window go to the General tab.

Do the following settings:
ResultSet : Single row.

ConnectionType : OLEDB(that’s what I have used you could also use ADO.NET)
Connection : Select the appropriate one.
Put the query to get the single file name from the table.



Now select the Result Set tab

Click the Add Button.

You will see the below screen with a row for the result details.

In the Result Name set it to : 0 (Numeric zero)

In the Variable Name select the string type variable which has the scope set to the package.

Once this is done click OK.




Now select the Flat File Connection Manager 1 and click "F4" or right click and select “Properties”.




In the properties pane for the above connection manager:

Set the DelayValidation to True

Expand the Expressions by clicking the ellipse.




On the Property Expressions Editor window:

In the Property column select the ConnectionString property from the drop down list.

Now expand the Expression column by clicking the ellipse. An Expression Builder window will pop up.




On the Expression Builder window:

Set the Expression as:

“C:\\Data\\” + @[User::FileName]

Click the Evaluate expression button at the left hand bottom to see if the expression is creating the desired output.

Click OK.

Click OK on the Property Expression Editor as well.




Now set up the Data Flow Task as below:




Double click the Flat File Source.

In the Flat File Source Editor:

In the Connection Manager tab, select the appropriate Flat file connection manager from the drop down list.




Now in the OLEDB Task:

Select the OLEDB Connection needed to connect to the destination Data base.

Select the table where the columns need to be inserted.




Goto the mapping

Go the mapping and do the mapping as required and click OK.




Save and build the package.
Now when you execute the package the package will take the filename from the DB table.

This could also be modified by having the file name stored directly to a variable and make this variable configurable.

Very soon I will be posting on setting up file names dynamically for multiple files.

Please provide your valuable feedback.


Tags: SSIS, Expressions, Substring, FindString, MSBI,


Sudeep Raj
25 · 7% · 1508
3
 
0
Lifesaver
 
0
Refreshed
 
0
Learned
 
0
Incorrect



Submit

9  Comments  

  • Very well explained !!..Good work Sudeep !..

    commented on Jul 5 2011 2:10PM  .  Report Abuse This post is not formatted correctly
    Sana ali
    482 · 0% · 46
  • Good article.... can you please upload Packages and Data you are using for your samples ... it will be great

    commented on Sep 6 2011 8:44PM  .  Report Abuse This post is not formatted correctly
    vishalchitrala@gmail.com
    626 · 0% · 29
  • This is exactly what I had been searching for. The part I had been missing was the properties for the flat file connection manager: "Delay Validation" and setting the ConnectionString expression. Thanks so much for posting it in detail.

    commented on Nov 7 2011 2:01PM  .  Report Abuse This post is not formatted correctly
    KathyPDX
    2037 · 0% · 4
  • I'm extracting to a flat file. This almost worked for me. The resulting flat file does not have the '.txt' extension. 'Africa-SouthAfrica-Consolidated-CMMIS-Maintenance-Equipment-2011-11-09' should be 'Africa-SouthAfrica-Consolidated-CMMIS-Maintenance-Equipment-2011-11-09.txt' The connection manager has '.txt' in the file name: 'Equipment_Extract.txt'. Why doesn't the final file?

    commented on Nov 9 2011 12:34PM  .  Report Abuse This post is not formatted correctly
    clhudson47
    2037 · 0% · 4
  • @clhundson47: Check the file name that you have in the table. Does the file name contain .txt?

    commented on Nov 9 2011 1:27PM  .  Report Abuse This post is not formatted correctly
    Sudeep Raj
    25 · 7% · 1508
  • It does now. Thanks!

    commented on Nov 9 2011 2:13PM  .  Report Abuse This post is not formatted correctly
    clhudson47
    2037 · 0% · 4
  • Sedeep, You said above "Very soon I will be posting on setting up file names dynamically for multiple files." Did you ever do that? I looked through your post titles and didn't think I saw it. Thank you.

    commented on Nov 21 2011 6:27PM  .  Report Abuse This post is not formatted correctly
    KathyPDX
    2037 · 0% · 4
  • Hi Kathy, Have a look at this: http://beyondrelational.com/blogs/sudeep/archive/2009/11/19/dynamic-file-connection-in-ssis.aspx

    commented on Nov 24 2011 9:35AM  .  Report Abuse This post is not formatted correctly
    Sudeep Raj
    25 · 7% · 1508
  • I have to dynamically create the SSIS connection string to a Flat File connection with a UNC path because if the computer specified in the 1st connection string is down, I want to continue running the package with the 2nd connection string. I have created a package scope string variable named ConnectionString with an initial value = \COS-FIDS-01\~fidsdatascript$\data.txt.

    I have a table that contains 3 rows, 1 for each computer and Execute SQL tasks to assign the values in the table to the string variable named ConnectionString. I have set breakpoints before and after the 1st Execute SQL task, with the value in the 1st row of the table = \AirTacknical\~fidsdatascript$\data.txt.

    When I inspect the breakpoint before the 1st Execute SQL task ConnectionString = \\COS-FIDS-01\~fidsdatascript$\data.txt, with every \ doubled to \

    When I step over the 1st breakpoint and execute the 1st Execute SQL task ConnectionString = \\AirTacknical\~fidsdatascript$\data.txt It looks like this is the required behavior for the ConnectionString variable to be assigned to the Flat File Connection Manager’s ConnectionString property using the Property Expressions editor as you show in your example. When I open the Expression Builder as you show above and click on Evaluate Expression I return \COS-FIDS-01\~fidsdatascript$\data.txt

    My package checks the connection string for the 1st row in the table using a File Properties Task I downloaded from CodePlex, but it appears to check using the value of the ConnectionString variable with the extra \’s

    How can I have another variable that avoids the doubling of the \ character so my File Properties Task can succeed?

    commented on Dec 1 2011 12:55PM  .  Report Abuse This post is not formatted correctly
    phorne
    2298 · 0% · 2

Your Comment


Sign Up or Login to post a comment.

"SSIS - Dynamically set Flat File Connection Manager" rated 5 out of 5 by 3 readers
SSIS - Dynamically set Flat File Connection Manager , 5.0 out of 5 based on 3 ratings
    Copyright © Beyondrelational.com Contact us      Privacy Policy      Terms of use      Report Abuse      Advertising