Custom Open Source Exchange Connector for SCSM
Since 2015 pretty much all of us who use System Center Service Manager (SCSM) have used the Microsoft Exchange Connector v3.1 to capture e-mails coming from end users and turn them in to Incidents. It works well and does what it says on the box…… But wouldn’t it be great if it did some other things?
What if it could merge replies to prevent multiple work items from being created, or work with encrypted e-mail systems, or even use AI to predict the subject and auto search KB articles for the end user?
All this and be open source so we could customise it ourselves?
That would be special…
Well that’s exactly what one member of the Cireson Community did!
Adam Dzyacky took on this challenge and has now created an Open Source, Community driven, PowerShell coded Exchange Connector that not only preserves the functionality of the Microsoft Exchange Connector but adds additional functionality.
Recently I was lucky enough to sit down with the connectors creator, Adam Dzyacky, and ask him a bunch of questions about the product so I thought I’d write a blog post to share with you some questions and answers including what was the genesis of this product, what are its goals, what are its current abilities and how can people use it today…. FOR FREE! 🙂
Question: Where did the idea of this connector come from and what was your thought process behind creating this connector?
Answer: Several years ago when I first got involved with Service Manager and the Exchange Connector I was immediately confronted with a problem – the stock connector only processes a single message type (IPM.Note). As such, any other message type is simply ignored. Out of Office, Meeting requests, Signed/Encrypted messages…all of it.
Ignored.
But hope was not lost because with some PowerShell and SMA, one could create scheduled SMA jobs to pick up what the stock connector missed. It would certainly introduce a new level of administration, but once it’s automated the work is done. I thought to myself;
“Well at least I can curb this with PowerShell so I guess it isn’t that bad.”
But I couldn’t help but shake the feeling that I can’t be the only one who cares about those other message types.
Next, if it wasn’t some new message type I’d have to deal with it was how the connector worked when it came time to process even those basic emails. Employees replying within a current processing loop of the connector, to the same thread of a message would generate new and unique Work Items for every single reply instead of simply appending to a single Action Log for a single Work Item.
Since the connector isn’t real time and instead every runs every X minutes…well a lot can happen between runs of the connector! It’s an unpredictable behaviour that requires the team(s) charged with that initial filtering to do a lot of Work Item micro-management thus detracting from their actual work of Resolving Incidents and fulfilling Service Requests. That’s potentially a lot of duplicate Work Items to close in SCSM and no less to understand to ignore in reporting.
In this case, supplementary PowerShell and SMA job can’t solve this because the Work Items have already been created. The connector would need to be able to understand the concept of an email thread at the source before Work Items are updated.
The above are but the first of many issues I had with the stock connector. It’s not that it isn’t great at what it does, it’s just I wished I could change some of it.
But no matter how much I wished I could change it the Exchange Connector is a sealed, closed source, C# management pack. Even if you could address this at its source, not only would you need an understanding of the C# programming language but you’d also need an in depth understanding of the System Center SDKs.
Question: So what was your plan of attack to fix these issues?
Answer: In February of 2017 I finally had enough of what wasn’t possible and committed my requirements to OneNote.
- Preserve all functionality of the stock connector
- Introduce some kind of new functionality over the stock connector
- Be modular to support new/changing processes
- Be open source
- No programming languages – need something more than just developers understand and could ultimately edit
Question: No programming languages? As an admin I love the thought of that. So what was the plan of attack?
Answer: So from here, the decision was straightforward. Build an Exchange Connector written entirely in PowerShell leveraging the widely used community PowerShell module that is SMLets.
On top of that, host on GitHub so that bugs can be tracked, features requested, and anyone can contribute.
If successful you’d be able to drop the stock Exchange Connector, improve performance on your workflow server (especially if you had multiple connector for multiple inboxes), optionally move the script into an SMA or Azure Automation RunBook, and of course introduce a host of new possibilities as the only limitation to new features would be PowerShell.
As per Tom Hendrick’s comment in the Cireson Community thread;
“Limitation and PowerShell do no often appear in the same sentence.”
Question: How long did it take you to write the initial version?
Answer: In what probably totals about three weeks of actual focused work – I had the first version done.
Question: Being Open Source means that anyone can contribute to it, but allowing people to contribute and finding people to contribute are two different things. Have you been able to garner support from others to help develop this solution?
Answer: Starting April 2017 I shared this with Tom Hendricks, Brian Wiest, Martin Blomgren, and Leigh Kilday who were gracious enough to provide their time to test and provide feedback for the first release published on GitHub later that month.
Question: So what exactly does it do? What are It’s features?
Answer: The connector has all of the regular features of the stock Exchange Connector plus new features that fall in to two categories:
- People who are using SCSM by itself
- People who are using SCSM with Cireson products
Features if you’re just using SCSM
More keywords
- Change Requests
- [hold]
- [cancel]
- [take]
- Incident
- [take]
- [reactivate]
- Problem
- [take]
- Service Request
- [take]
- [hold]
- [acknowledge]
- Manual Activity
- [skipped]
Announcements
Just throw [announcement] in your next email to Service Manager and as long as your part of a configurable AD group that’s defined an Announcement will get created in SCSM. Need to control the priority? Just add an additional #low or #high. Announcements default to normal priority otherwise. And yes, you can update announcement simple by keeping the [Work Item] in the subject.
Minimum File Attachment Size
No more signature graphics as attachments. Set a minimum like 25kb and your Work Items will get a whole lot cleaner.
Maximum File Attachment Size
Optionally enforce File Attachment Settings as defined in the Administration -> Settings pane of each Work Item type.
File Attachment “Attached By” Relationship
When the connector processes messages, the Sender will be marked as the “Attached By” relationship for attachments. This is useful when multiple parties are updating the same Work Item.
Review Activities without [approved] or [rejected]
Do your end users think someone is actually reading the Service Manager inbox so they respond with questions to RAs? Fret not because now comments that don’t contain a vote will get appended to the Action Log of the highest Parent Work Item
Vote on Behalf of AD Groups
Open up a whole new world of voting possibility!
Schedule Work Items
The Scheduled Start/End times of a Work Item can now be set by sending a Meeting request to Service Manager. No Work Item yet? Just like email, if a Work Item doesn’t exist to update a new one will be created only now those date fields will be set in addition to the Work Items creation.
Digitally Signed/Encrypted Messages
Leveraging the open source MimeKit project the connector can process digitally signed or encrypted emails just like regular mail.
SCOM Integration
Get the health of your [Distributed Apps] and their current Active Alerts.
#private replies
Want to keep the notes between analysts? Just throw in a #private in your message to SCSM and it’ll get marked as Private on the Action Log.
Merge Replies
No more duplicate Work Items because now when users Reply to an email that does not have a [Work Item] in the subject, Service Manager will identify the email thread they were in and update the one, true, correct Work Item.
Create Related Work Items on Closed Work Items
Sometimes employees send an email about a Closed Incident. Rather than turn a blind eye, a New Related Work item will get opened for them and copy information from the previous Work Item into the new one along with their recent comment.
Multiple Inboxes
Configured correctly, you can redirect several inboxes on Exchange to your single Service Manager inbox. On top of this, unique templates can and will still be applied based on the source inbox they were redirected from. Buh bye multiple connectors!
More Default Work Item Types
No reason to limit yourself. The connector can now be configured to created Change Requests or Problems by default. Great for vendors sending maintenance or analysts generating Problems.
Artificial Intelligence
Did you battle with classic Exchange Connector dilemma of “What should the default work item type be when people send in emails – Incident or Service Request?” Wouldn’t it be great if Service Manager could just decide whether or not it should create an IR or SR based on the Affected User’s perceived attitude? Thanks to Azure Cognitive Services, emails can now be run through Sentiment Analysis and based on the rating will dynamically create either a Service Request or Incident based on a minimum defined score as configured per organisation.
Features If You Are a Cireson Customer
Suggest Knowledge Articles
You can optionally enable the connector use the body of the email as a search query against one’s respective Cireson HTML KB. Once complete, the connector will send an HTML email back to the Affected User with suggested Knowledge Articles and hyperlinks to them.
Suggest Knowledge Articles
You can optionally enable the connector use the body of the email as a search query against one’s respective Cireson Service Catalog. Once complete, the connector will send an HTML email back to the Affected User with suggested Request Offering and hyperlinks to them.
Send Outlook Meeting
The connector supports the ability to create or update Work Items from Meeting Requests. This introduces a New Work Item task on the Cireson portal so you can further leverage this feature.
Announcements
Just throw [announcement] in your next email to Service Manager and as long as your part of a configurable AD group that’s defined an Announcement will get created in the Cireson SCSM Portal. Who will see it? Simple – the Distro groups you included on your email message out! Need to control the priority? Just add an additional #low or #high. Announcements default to normal priority otherwise. And yes, you can update announcements simple by keeping the [Work Item] in the subject.
[take] Keyword Restrictions for Support Groups
Maybe you want to put some restrictions on who can [take] things. Leveraging the Cireson Web API this is now possible checking to see if the Sender is part of the Support Group the Work Item is currently assigned to.
Artificial Intelligence
Instead of using the entire email body to suggest Knowledge Articles or Request Offerings to the Affected User, Azure Cognitive Services will pick out the keywords of the message and use those words to drive suggestions. This results in more focused searches and faster processing times.
Question: WOW! That’s a lot. What’s next on the planning table and how can others join in the conversation?
Answer: A few that come to mind are things like creating Work Items on behalf of others through the connector, assigning to yourself on Create, and as GitHub community suggested – integrating with the Cireson Portal Watchlist feature. All of these can be found on the repo’s Issue page.
Speaking just for myself I’d say that since day 1 I’ve wanted some kind of AI integration and fortunately Azure Cognitive Services readily provides that through easily consumable APIs. While we have sentiment and keyword analysis in the current version, I think the more interesting topics are things like their using their Speech API to convert voicemails to Work Item descriptions or using LUIS to understand intent to drive specific actions within SCSM. But ultimately, just discussion at this point.
Question: How would someone get involved in contributing to the project if they wanted to?
Answer: All it takes is a GitHub account. After you sign up you can Fork the repository. This, in short, creates a duplicate SMLets Exchange Connector under your own account that you can edit and change how you see fit and submit requests to Merge back into the master repository if you want. Cireson Community member Roland Kind has done this to start building a version that makes use of the stock SCSM cmdlets if you prefer that module instead.
An account also gets you the ability to suggest features, post bugs, and join the conversation directly on the Issues page. Maybe you just want to be notified when there are changes? If you put a Watch on the repo you can get email notifications when changes occur. Or if you just want to show your support you can also Star the repository.
Conclusion
The new PowerShell based Open Source Exchange Connector is nothing short of AMAZING!
Thanks go to Adam Dzyacky and anyone else who has contributed to this solutions for all the hard work and dedication to get this solution up and running.
New features get added regularly and there is a vibrant and energetic group of contributors who keep it updated and supported. (Not sure I could say the same about the MS Exchange Connector offering – Last updated in 2015)
While some organisation may have issues with this solution being Open Source and not officially supported by a vendor, I personally think the benefits far outweigh the possible risks. Considering the time and effort we all spend micro managing the results of the out of the box connector this new solution will shave tens of hours per week in support effort.
System Center Noise 