One day, while you are frantically attempting to regain control of your SQL Server Integration Services (SSIS) package’s conversion error, where you swear that an input date column does NOT contain ‘out-of-range values’, the Head Of Data approaches you and asks if you know about Data Lake. You have heard whispers about this technology from who now seem to be the most innovative people in the company, the Analysts, but you did not have time to read up about it because you were drowning in Change Requests. So you reassuringly reply with “I am aware of it”.

“Excellent”, says your Head of Data with glee. “We are looking to migrate all of our data into the Cloud, and I want you to do it using Data Factory (ADF)”.

“No prob boss” are your nervous final words as you immediately scramble to open Google. After a few searches of ‘Data Factory’, you are in fact quickly comforted by seeing the terms ‘no code or maintenance’ and ‘visual environment’, which were exactly what enticed you to SSIS in the first place. Now out of curiosity, you decide to blow the dust off your never-used company provided monthly Azure credit and create your first Data Factory. As the resource opens, you immediately look for familiarity:

  • Toolbar on the left… “check”
  • Visual surface… “check”
  • Components from the toolbar drag onto the visual surface with a mouse… “check”
  • Components remain where you place it rather than fly to the top left of the editor… “wow, there’s improvement over SQL Server 2012!”

You are pleased with the outcome so far, but now it is time to build something. At this point you hesitate and panic begins to set in when you start asking yourself:

  • “What are pipelines?”
  • “What are these Activities?”
  • “Where is the connection manager?”
  • “Where are the input parameters?”
  • “Oh look I found an ‘Execute SSIS Package’ activity perhaps I can just continue as an SSIS developer?”

Fear not… the dream of becoming an Azure Data Engineer is not over. As you can probably tell, I have described a somewhat dramatized version of my transition journey between the two ETL tools. You will be pleased to know that the majority of the abilities you happily managed within SSIS are achievable in ADF, and hence I would like to take you through these in my series of blogs ‘From SSIS to ADF’.

Oh… and to try and answer your last question of if it is better to continue using SSIS, I would say there will be less need for SSIS once ADF’s Data Flow is out of preview.