Category: Connectors

How to use the Cireson Asset Import Connector

A little while ago on the Cireson Community Forum a member asked for more details on how the Cireson Asset Import Connector works. So I decided to write a blog post about it to clear up exactly what the connector is and how it works. I also recorded a short video for those of you who do not like long winded blog posts. You can find the video here.

The Cireson Asset Import Connector is one of the solutions contained within the Cireson Asset Management Stream of products and allows for Asset Administrators to take the guesswork out of importing external data into System Center Service Manager. This app allows any out-of-the-box CMDB data, or any information in the Cireson Asset Management app, to be imported from external CSV, SQL, ODBC or LDAP sources of truth, exposing an intuitive interface that provides the ability to map columns and schedule imports when required.

All little know pub quiz fact is that the Cireson Asset Import App grew from the CSV import app which was the very first Cireson app to hit the market. Next time this question comes up in a pub quiz, rest easy knowing that you now have the answer and are in a pub that is so cool it asks question like that one! ūüôā

When you add the Cireson Asset Import app to a Service Manager environment, importing data becomes seamless. One-time imports and configuring XML files become a thing of the past. The straightforward app provides the organization with the ability to build an asset repository of information that is relevant and accurate when working with requests in Service Manager.

So lets get in to it… throughout the following post, I will call out important things to note and also what is generally regarded as “Best Practice” but always consider the¬†requirements and impact these settings may have.

1. Creating a new Asset Import Connector

  1. Within the SCSM console, select the Administration workspace.
  2. Right click the Connectors Node.
  3. Select Create Connector from the drop down menu.
  4. Select Asset Management Import Connector from the sub menu.
 ami01
 ami02 NOTE:

The sub menu option for Asset Management Import Connector (Import) is for creating pre-created or backed up Import Connectors.

Enter a name for the connector that will make sense to other administrators for future maintenance tasks.

Select a Management Pack (or create a new one) that will be used to contain the workflow information required for the workflow of the connector.

 ami03
 ami04 Cireson Best Practice:

Best practice for creation of Management Packs is to create these Management Packs via the SCSM authoring tool and giving it an internal and full name in the format of ‚Äú ‚Äď Asset management Import Connectors‚ÄĚ.

This then assists to identify the Management Pack when exported or backed up at a later date.

The next step will be different depending on the input data source. Select and use one of the following sections below before continuing.

2. Using a CSV Source

After completing the steps in the section below, browse to the location of the .CSV file that contains the asset data to import and select the Encoding Format of the file.

The selected path can be either a local path (on the SCSM workflow server) or a network share that has read permissions by the Workflow account.

The first line of the CSV file must contain the header row information for the data contained within.

 ami05
 ami04 Cireson Best Practice:

It is Cireson best practice to create a single folder that contains all the CSV import files for any connector that is being used. It is also best to configure the connectors to use a UNC path as the location path of the file selected as this allows the connector to be edited successfully from other computers.

 Continue the connector settings.

 3. Using a SQL Source

For Microsoft SQL Server data source:

Enter the SQL Connection string by clicking the ellipse button and entering the required connection information.

 ami02 NOTE:

If Windows Authentication is to be used, the SCSM Workflow account must have read access to the source database.

Enter the SQL query that will be used to extract the data required for this connector.

Click Execute Query to test the query and gather field name requirements for class property mapping.

The SQL Query Results field will show the number of row returned if the query was successful.

 ami06
Continue the connector settings.

4. Using a ODBC Source

For ODBC Server data source:

Create a File Data Source Name (DSN) that contains the Server, Database and username for the data source.

Browse the file system and select the File DSN.

 ami02 NOTE:

The SCSM Workflow account must have read access to the File DSN.

Enter the File DSN Password for the username within the File DSN.

Enter the SQL query that will be used to extract the data required for this connector.

Click Execute Query to test the query and gather field name requirements for class property mapping.

The SQL Query Results field will show the number of row returned if the query was successful.

 ami07ami08
Continue the connector settings.

 5. Using an LDAP Source

For an LDAP data source:

Enter the LDAP Server or Namespace and the LDAP Port (If required).

If the SCSM Workflow account does not have read access to the LDAP source, enter alternative credentials with the required rights.

Enter the LDAP Attributes that are required to be returned separated by commas.

Enter an LDAP search starting path to reduce the search scope as required.

Enter any LDAP Filter needed to refine the results to the specific required data.

Click Execute Query to test the query and gather field name requirements for class property mapping.

The LDAP Query Result field will show the number of row returned if the query was successful.

 ami09ami10
Continue the connector settings.

6. Connector Settings

Select the target class that the records will be imported in to. This might be one of the base classes (Such as Hardware Asset) or, if other relationships are required, selecting a combination class (Type Projection) that contains the relationships required for the import.

Enter a Workflow log path to track import results and reporting on success\failure.

 ami11
Set the required options for the instance of the Asset Import connector. See below for more details on these options.

Once all options are selected, click Next.

 ami12
Asset Import Connector Options:

Test Mode The connector will run and create log file for inspection without commiting any changes to the SCSM database.
This connector can create new items When enabled, this option will allow the connector to create new records within the database.

This is used to allow the import of new records.

This connector can update existing items When enabled, this option will allow the connector to update existing records that match the key fields the selected class.
This connector will DELETE ALL matching items only This option changes the behaviour from creation to deleting of records. Any record matched from the import data to an instance of the class will be removed from the SCSM database.

WARNING! If data is deleted it can not be recovered.

This connector will update multiple existing items matching specific custom keys
Do not replace \n with a linefeed By default, the improt connector will interperate any \n text as representing a new line and therefore will replcae it with a linefeed character within SQL.

7. Mapping Fields

Data Mappings allow the mapping of the specified input data to the properties of the selected target class within SCSM.

On the Data Mapping screen, if the option for ‚ÄúThis connector will update multiple existing items matching apecific custom keys‚ÄĚ is selected on the previous screen the first option that will show is for Custom Keys. Custom Keys are used to fins all existing matching items and update them as normal via the mappings below. At least one custom key is required.

The Custom Key can be any of the properties for the class that was selected for this connector.

Add the custom keys as required and map these to the data from the import source.

 ami13
 ami02 NOTE:

All Key Properties for the selected class as well as any Custom Keys are required fields and must be mapped to continue.

The property displayed in the left column will show all properties of the selected class, along with any extended properties that have been added for the class.

The Data Type in the middle column will show what input data type the property will expect. String (Key) identifies the primary key for the selected class.

The Mapped To value displayed in the right column will show drop-down values for each available column header from the specified source

The Hardware Asset ID should be mapped to the primary key selection you chose in the Asset Management Settings. (Serial Number, Asset Tag, GUID, etc.)

Map all additional properties to the input data that is defined from the Input source.

Any properties that are mapped will be updated or entered as defined.

Any properties that are not mapped will not be updated.

 ami14
If a Combination Class is selected for the connector there will be additional mapping fields under the Relationship heading.

These can be used to map data from multiple classes together as relationships as required.

 ami15
Once all mappings are complete, click Next.

8. Connector Workflow Schedule

Some connectors will be run as a once off to import bulk data in to the SCSM database, whereas others might be run on a schedule to keep other data sources up-to-date within the database.

An example of a scheduled data source might be a connector in to a Mobile Device Management (MDM) solution or an accounting or purchase system (for invoices and Purchase Orders).

For connectors that will be only run once, select the option marked This connector will be run manually.

When using this option, a warning message will be displayed to remind administrators that the connector will only run when using the Synchronize Now task within the console.

For a reoccurring schedule, enter the frequency as either daily or as a regular reoccurrence with a set frequency.

Ensure the Connector Enabled option is enabled to all ow the connector to run. This option may help with the administration of the connector at a later date if it needs to be turned off for a period of time for maintenance or fault finding.

 ami16
When the scheduling information has been entered, click Create.  ami17

9. Manually Running a Connector

Once a connector has been created it will show within the Connectors node in the Administration workspace of the SCSM console. Within this node, administrators are able to see the current status of all connectors, when they were last started and finished and their percentage complete.

Administrators are also able to manually run a connector to either force the synchronization regardless of workflow schedule or to trigger a non-repeating connector.

To manually run a connector:

Within the SCSM console, select the Administration workspace.

Select the Connectors node.

 ami18
Select the Connector to be run and click the Synchronize Now task within the tasks pane.  ami19
If the connector does not have a schedule set (is disabled) then a message will appear informing that the connector is disabled and asking if it should still be run.

Click Yes to run the Synchronization.

 ami20
The connector workflow will then be scheduled to start at the next opportunity for the workflow engine.

10. Exporting and Importing a Connector

Once a connector has been configured the settings can be exported to allow administrators to copy the connector to a different environment (dev to prod).

To export and import a connector:

Within the environment to export from:

Within the SCSM console, select the Administration workspace.

Select the Connectors node.

 ami21
Select the Connector to be run and click the Export task within the tasks pane.

Save the connector XML file to a path and click Save.

 ami22
Within the environment to import in to:

On the Connectors node, select Create Connector from the drop down menu.

Select Asset Management Import Connector (Import) from the sub menu.

Browse to the folder containing the exported XML file, select the xml file to import and click OK.

 ami23
A window will appear to rename the Connector from its original name if required and change the Management Pack that holds the information.

If the connector is importing from a CSV file, an additional field will appear that is used to provide the source location of the CSV file required.

Enter the values needed and click OK.

 ami24
The connector will be imported and will now appear in the connectors node.

11. Deleting a Connector

If a connector is no longer needed, then it can be removed from the SCSM environment by deleting the connector from the console.

To delete a connector:

Within the environment to export from:

Within the SCSM console, select the Administration workspace.

Select the Connectors node.

 ami25
Click the Delete task from the tasks pane on the right of the screen.

Click OK on the message that appears to confirm the connector to be deleted.

The connector has previously imported data a second message will appear asking if the data that was imported from the connector should be deleted.

 ami26

Hope this gives you a clear idea of how this app comes together and works for your organization.

Leave a comment if you have any additional questions.

 

Advertisements

Runbook is in an Invalid State

PROBLEM


A common issue I run in to a lot with SCSM automation is the Following error message:

The Runbook associated with this Runbook activity template <Name of template>, is in an invalid state. Select another Runbook or ensure that the Orchestrator connector is properly configured

The Runbook associated with this Runbook activity template <Name of template>, is in an invalid state. Select another Runbook or ensure that the Orchestrator connector is properly configured

Error message in the SCSM console

CAUSE


This is caused by the Runbook being in an invalid state within SCSM, not within Orchestrator.

To see what I mean, within SCSM, Navigate to the Library workspace and select the Runbooks node.

Invalid Runbook

Invalid Runbook

When a Runbook within SCSM is in an invalid state, it is usually because the input Properties for the Initialize activity within the Runbook itself has been changed since the first sync of the Orchestrator connector and SCSM does not know what to do with the new properties. (or the removal of the old ones)

SOLUTION


The solution is fairly straight forward.

Within the SCSM Console, select the Runbook that has a status of “Invalid” and select Delete.
This will delete it from the SCSM Console and not Orchestrator.

Then re-run the Orchestrator Connector:

  1. Select the Administration workspace
  2. Select the Connectors node
  3. Selecting the Orchestrator connector you need to re-run
  4. Click Synchronize Now in the tasks pane

Re-run the Orchestrator Connector

Re-run the Orchestrator Connector

Once the connector has finished it should all be back to normal.

Error: “An Error Was Encountered While Running the Task” when Creating a Connector in SCSM

While creating an AD connector in SCSM2012 R2 A few months back I got the following error:

Server request failed for command 3
Exception type:TargetInvocationException
Exception message:Exception has been thrown by the target of an invocation..
StackTrace: at System.RuntimeMethodHandle._InvokeConstructor(Object[] args, SignatureStruct& signature, IntPtr declaringType)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Security.Cryptography.CryptoConfig.CreateFromName(String name, Object[] args)
at System.Security.Cryptography.Rijndael.Create()
at Microsoft.EnterpriseManagement.ServiceManager.Connector.Datacenter.DatacenterClass.a02(Byte[] s, Byte[] k, Byte[] v)
at Microsoft.EnterpriseManagement.ServiceManager.Connector.Datacenter.DatacenterClass.a01(String s)
at Microsoft.EnterpriseManagement.ServiceManager.Connector.Datacenter.DatacenterClass.RegisterDataSource(DataSourceObject ds, Int32 solutionId)
at Microsoft.EnterpriseManagement.ServiceManager.Connector.SessionManager.DataProvider.RegisterDataSource(XPathNavigator node, Int32 connectorId, Int32 groupId, Boolean enabled, Int32& dataSourceId, String& filter, Boolean& notify)
at Microsoft.EnterpriseManagement.ServiceManager.Connector.SessionManager.DataProvider.RegisterSyncSchedule(XPathNavigator node)
at Microsoft.EnterpriseManagement.ServiceManager.Connector.Datacenter.DatacenterClass.Configure(XPathNavigator xpath)
at Microsoft.EnterpriseManagement.ServiceManager.Connector.Datacenter.DatacenterClass.ApplyDataSyncByPropertyBag(Dictionary`2 propertyBag)
at Microsoft.EnterpriseManagement.LinkingFramework.LinkingFrameworkServerRequest.doUpdateDataSource(Guid g)
at Microsoft.EnterpriseManagement.LinkingFramework.LinkingFrameworkServerRequest.createDataSource(Guid g)
at Microsoft.EnterpriseManagement.LinkingFramework.LinkingFrameworkServerRequest.LinkingFrameworkRequest(Int32 commandId, ResultSet values)

Inner Exception:
Exception message:This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms..
StackTrace: at System.Security.Cryptography.RijndaelManaged..ctor()

 

The connector was created but would not run and when I tried to delete the connector I got an error:

An error was encountered while running the task

The key to this error is the last line of the error message when creating the connector that states:

Exception message:This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms

Network Steve touched on the error and the solution in his Blog Post: http://www.networksteve.com/forum/topic.php/SCSM_Active_Directory_Error/?TopicId=38758&Posts=0

However, I wanted to explain this a little more.

What is FIPS?

The United States Federal Information Processing Standard (FIPS) 140 standard defines cryptographic algorithms approved by the US Federal government for use on their computer systems. Implemented approved cryptographic algorithms are considered FIPS 140-compliant only if it has passed validation by the National Institute of Standards and Technology (NIST).

Enabling FIPS mode within your environment makes Windows and its subsystems use only FIPS-validated cryptographic algorithms. Enabling FIPS mode also causes the .NET Framework to disallow the use of non-validated algorithms.

For more details on what FIPS is check https://en.wikipedia.org/wiki/FIPS_140-2 and http://technet.microsoft.com/en-us/library/cc750357.aspx

Microsoft originally recommended in its Security Baselines that FIPS should be turned on however have revised their recommendations http://blogs.technet.com/b/secguide/archive/2014/04/07/why-we-re-not-recommending-fips-mode-anymore.aspx

Where do I tell if it is Enabled?

On the management server from which you are creating the AD Connector;

First check the registry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\FipsAlgorithmPolicy

If the key has the Enabled value set to 1 then it is Enabled and you can change this value to 0 to disable the policy.

If this key has the Enabled value set to 0 then it is Disabled and the policy is being effected elsewhere, probably via Group Policy.

To check the group policy state, open the Local Security Policy Editor  and open the Local Policies > Security Options node and search for the policy titled “System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing

Capture2  Capture

If this is enabled then the policy is being set via Group Policy. Speak to your policy admins about setting and exception for this server.

One last piece is that once the connector has attempted to be created and fails, it can not be deleted via the console. The only way to get rid of the connector is via good-ol’ PowerShell.

Capture3