Skype4B interoperability: Cisco Expressway

Cisco Expressway “offer users outside your firewall simple, highly secure access to all collaboration workloads, including video, voice, content, IM, and presence. Collaborate with people who are on third-party systems and endpoints or in other companies. Help teleworkers and Cisco Jabber mobile users work more effectively on their device of choice. Cisco Expressway allows you to do all this and more.” (source)

Using Skype for Business terminology, this is the “Edge Server” of Cisco. But it besides the described functions, it has one that allows you the integrate Cisco Video endpoints (desktop client, video conference rooms up to telepresence system) with Lync/Skype for Business clients that allows SIP dialling from both sides with audio and HD video: search for the contact and press the call button.

 

The infrastructure configuration is based on a ‘dedicated expressway’ configured as a Skype for Business ‘trusted application’. The rest is all about SIP domain ‘static routing’.

CiscoExpressway-Interoperability
Expressway topology for on-premises Skype for Business and Cisco video endpoints 

Great features of this integration:

  • IM & presence is possible between systems;
  • Desktop sharing is supported;
  • Both Cisco and Microsoft can share a single SIP domain;
  • Skype for Business remote clients can use the Expressway on the DMZ (TURN server) for audio traffic with Cisco endpoints;
  • Skype for Business clients can make and receive audio/video calls with external parties whom the Cisco has federation with.

There are also some limitations:

  • Presence is based on SIP simple. Only can have the status ‘Available’ and ‘Busy’;
  • Skype for Business clients can join a multipoint conference in Cisco, but Cisco endpoints cannot join Skype meetings.

 

In conclusion, although it’s not a better solution than Polycom, the Expressway leverages the existing investment  already done on Cisco infrastructure and can be extended to a complete 3rd interoperable federation solution:

CiscoExpressway-Federation

Lync (Skype for Business) November 2016 update – duplicate IM’s

ISSUE

On November 2016, Lync 2013 (Skype for Business 2015) customers start reporting cases of  duplicate of IM messages.

CAUSE

The issue started appearing after the installation of November 2016 client update (KB3127934) and it affects the receiver only.

skyp4b-kb3127934-issue3

duplicate-imsSteps to recreate the issue (credits go to Alex) :

  1. Send an IM to an user
  2. Do not open the IM toast on the receiving user
  3. Send another (or more) IM the that user
  4. Open the notification toast of the receiving user and you will noticed that only the first IM line sent is not repeated

The issue does not occur when the IM conversion window is open.

Applying the December 6, 2016, update for Skype for Business 2015 (Lync 2013) (KB3127976) also causes the same issue.

Skype for Business 2016 clients are also affected on the same way by their corresponding monthly cumulative updates. There is an oldest thread on Microsoft community that was reopened by another 2 persons that report the same behaviour and uninstalling the KB3127934 would solve the issue.

SOLUTION

Apply the January 3, 2017 update for Skype for Business 2015 (Lync 2013) (KB3141468) or Skype for Business 2016 (KB3128049).
It is mentioned on the resolved issue list (no root cause provided):
“Assume that you send continuous instant messages (IMs) to a user in Microsoft Skype for Business 2015 (Lync 2013). Then you allow the toast notification window to be auto accepted. In the conversation window, you find every item but first gets duplicated. Also, in the Conversation History in Microsoft Outlook, you find that the conversation window shows duplicated IMs. “

I successfully tested and confirmed that it solves.

Skype4B 2015 quick tip: keep debugging tools automatically updated

As you know the Debugging Tools is a separate installation product published a few weeks/months from the initial Lync/Skype4B. This tool is fundamental on troubleshooting the platform.

debugging-tools

It contains the information required to decode the debug traces of every component on two files: ‘default.tmx’ and ‘default.xml’, included and installed by the debugging tools package.
But they are also on the Lync/Skype4B package installation and all cumulative updates. Every new update/feature might require the ‘decoder’ to support the updated component. So, if you install the debug tools they are outdated and you might not be able to decode new features, partial logs line or even none.

The information on how to update them it’s referenced on the main cumulative update page  KB2809243 for Lync 2013 (doesn’t exist for Skype for Business 2015, but it’s the same principle):
Debugging tools require the latest version of the Default.TMX file that is included in each Cumulative Update to properly decrypt logs files. In order to keep … Debugging Tools updated, you will need to browse to the “C:\Program Files\…\Tracing” folder, and copy the default.tmx and default.xml files to the install location of Lync Debugging Tools. The default location is C:\Program Files\…\Debugging Tools\.

But there’s a much efficient and automated way to do this. Instead of copying the 2 files on every cumulative update, just replace them with an ‘symbolic links’ to the main Lync/Skype4B location:

  • Delete the default.tmx, default.xml file on the debug tools installation folder;
  • Create a symbolic link for each file (command line), ex:
    MKLINK “<Debugging Tools install folder>\default.tmx” “%CommonProgramFiles%\Skype for Business Server 2015\Tracing\default.tmx”
    MKLINK “<Debugging Tools install folder>\default.xml” “%CommonProgramFiles%\Skype for Business Server 2015\Tracing\default.xml”

debugtools-tip

Note: do not mix ‘symbolic link’ with a ‘shortcut’. The debug tools (and any other application) will not support the second option.

Every time the you run cumulative update package, the debugging tools will be pointing to the most up-to-date (for sure).

Hope you can find this simple trick useful. 😉

No Lync 2013 server updates available

If you are looking for a cumulative update for Lync 2013 servers you will not find any available since 6 December 2016.

I will find the download page KB2809243 with the following message:
An issue was discovered in the Lync Server 2013 November 2016 Update (build 8308.974) that causes contact searches on mobile clients to return no results. Because of this issue, the November 2016 Update is no longer available for public download. The Skype for Business team is working on a fix that is scheduled to be delivered soon in a new update.

Which complicates your Sysadmin life, since there were some the important fixes/updates:

  • KB3204553 – Lync Server 2013 adds support for Skype for Business for Mac
  • KB 3204552 – Skype for Business mobile clients don’t show telephone numbers for some users on contact card
  • KB 3204547 – You can’t join a meeting from Safari or Firefox through a Lync Server 2013 Mac app
  • KB 3204546  – (again this one) You can’t join a meeting from outside Skype for Business or Lync on iOS 10.0 and later versions

Update (13/12/2016): The cumulative update is available again for download. It has an updated published date, but no information that it replaces the one released two weeks ago.

Update (14/12/2016): Microsoft release KB3212869 identifying the November 2016 CU (8308.974) issue whose solution is to download and deploy this undocumented CU (8308.977)
I guess that if you ran it, will update any previous installed version… and one more ‘extra patching planning’ to perform.

The smallest Skype for Business front-end server

There were some reasons that took me a week on this project:
– I have few resources on my personal lab (specially storage);
– take my knowledge on OS, Skype4B deployment to the limit.

 If I already ‘get on the nerves’ of customers and colleagues when I request and deploy servers with less then 100GB of HDD,  I can imagine Microsoft with the minimum requirements of 72GB of free disk space (not including the OS?).

My current standard uses a Windows 2012 server with a total of 55GB split between 3 HDD. The Operating System (Drive C) and Skype for Business Front-End (Drive D) take around 36 GB.
skype4b-win2012-size

How much smaller can you have the same Front-end server?  around 18GB (*)
skype4b-win2012r2core-size
(*)not counting the space for IIS logs, Windows Fabric traces and the Page file

The answer for this the same for the question: Can I deploy Skype for Business on a Windows 2012 R2 server core?
Here’s some good reasons to use the Windows server core edition as Microsoft describes:
– less disk space and ram consumption;
– Reduced attack surface (no GUI and less OS vulnerabilities).

In fact the core edition has 98% of the installation prerequisites for Skype for Business Server 2015. On this post I will enumerate the challenges you face if trying to do these. Some are real challenges, others are just glitches of the main Skype4B setup.

Windows Identity Foundation 3.5 (WIF)

This is one prerequisite that you will get an error, and Microsoft KB clarifies that you will not be able to install without installing 4GB of the minimal server interface. All this to get 7 small outdated files that are supposed to be included on the .Net framework 4.5 (included natively on Windows 2012 R2 Server).
In fact that is even described on the OS package:
Microsoft-Windows-Identity-Foundation-Package~31bf3856ad364e35~amd64~~6.3.9600.16384.mum: “Windows Identity Foundation (WIF) 3.5 is a set of .NET Framework classes that can be used for implementing claims-based identity in your .NET 3.5 and 4.0 applications. WIF 3.5 has been superseded by WIF classes that are provided as part of .NET 4.5. It is recommended that you use .NET 4.5 for supporting claims-based identity in your applications.

NOTE: bootstrapper.exe doesn’t validate if WIF is installed on the prerequisites stage. You will only get an installation failure at the package MicrosoftIdentityExtensions.msi.

The workaround is about being able to ‘add-package’  above 😉

IIS Management console

This is a ‘strange prerequisite’. Why do you need the IIS management console snap-in (MMC) to install/run a Skype4B ?
missing-iis-mmc
MMC support is only available installing the  minimal server interface or you will get an error when trying to install it: Add-WindowsFeature Web-Mgmt-Console.
Workaround: just provide the key that bootstrapper looks for by adding the REG_DWORD  value ‘ManagementConsole‘ to the ‘HKLM\SOFTWARE\Microsoft\InetStp\Components‘ key. You can even set to zero (not installed) since it only checks for its existence.

Media Foundation

This is a little more ‘ridiculous’. You can install media foundation on windows core:
dism.exe /online /enable-feature /featurename:ServerMediaFoundation /all
but even it appears as installed on the get-windowsfeatures, the bootstrapper will report missing
missing-mediafoundataion

The reason is that it’s checking for a different installed component: ‘Server-Gui-Shell‘ which is another additional extra to the  minimal server interface

Workaround: add the REG_DWORD value ‘Server-Gui-Shell’ (must be 1) to the ‘HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Server\ServerLevels‘ key.

The last ‘twist’

By this moment you managed to install all the Skype4B front-end components. You managed to start the main service (RTCSRV) but the ones who rely on audio (ex: RTCAVMCU, RTCCAA) and remote data access (ex: RTCDATAMCU). The reason is that 7 required dll’s are only included on the Windows server standard edition:
– DirectX11 and real-time media handlers;
– Remote Access handling.

Workaround: as soon as you get a copy of the missing 7 dll files, you manage to start the remaining Skype4B services and you now have a fully operational Front-end server!

Conclusion

From the description above, the big reason that Microsoft doesn’t support Skype4B on a Windows server core is 7 dll files that are not able to be separated from the install edition.

Off course by now, you can see that this is an option for functional testing in LAB or demos. Microsoft will never support this, even if there is a way to install all the missing parts using the several windows setup command lines available.

The other no-go would be the administration/operations team: There will be a ‘revolution’ if people find out that there was no Windows GUI to manage a server (although you can manage servers remotely with a full GUI ‘management server’.

As a last comment: using the ‘MS-approved’ Windows server, I will let you know that it’s possible to run using a Windows 2012 R2 with a little less than 30GB of HDD.
skype4b-win2012r2-size
…but there’s still room to squeeze a little more 😉

Lync/Skype4B embedded links exploit

I decided to share this MSitPros blog post to show how can you exploit a Lync/Skype4B rich IM, using embedded links with SMB shares.

careful
As stated by the author, exploiting for the NTLM hash might be less successful from an external attacker (SMB traffic blocking), but a rogue LAN user or a deceiving ‘hotspot provider’/’internet cafe’ might would try this one.

Rich text IM (rich fonts, embedded pictures and links) is a very nice feature of Lync/Skype4B but it is also where the common MS Office security issues are found:

  • MS16-039: Security update for Microsoft Graphics Component: April 12, 2016
  • MS16-097: Security update for Microsoft Graphics Component: August 9, 2016
  • MS15-116: Security update for Microsoft Office to address remote code execution: November 10, 2015

Don’t panic right way if you have a full control/security policies of your LAN users, so that no one can just plug a rogue device (or install the required exploit software on his work PC).
The attacker must be able to reach the user – either he has an internal Lync/Skype4B account (which means he already might have hacked the network), or using Company or Skype federation.
Even if the attacker get the hash, the next step is to use against a server resource to access. An external attacker will have an additional challenge to reach your internal LAN.

Just like using Outlook, be careful when opening links or attachments. Better ways to prevent this:
– block links on IM (at least for federations)
– use only the NTLMv2 or Kerberos authentication protocols (although there are known ways to exploit them the same way)

My keynote is that security is an important topic when planning and deploying Lync/Skype for Business… don’t just go for a plain next>next>ready installation.

Call Quality Dashboard – Part 3: The Portal

After describing the Call Quality Dashboard (CQD) QoE Archiving Database and the QoE CUBE, I will show now how to install the Portal component and how it works on the solution.

The CQD Portal is “where users can easily query and visualize QoE data.” synchronized by the Archive and processed by the CUBE.
ic841926The CQD Portal is a IIS based web application that allows you not just visualized but create new reports, views and assign permissions to them. As the above picture shows, it relies on a SQL database to keep all the information.

Installing CQD – Portal

Before performing the installation, the following pre-requisites need to be in place:

  • You need a SQL Databases Services (dedicated or existing) for the setup to install the Portal support database.
  • On the server that will host the Portal you need to install IIS. The following powershell command will install all the required components:
    Add-WindowsFeature Web-Server, Web-Static-Content, Web-Default-Doc, Web-Asp-Net, Web-Asp-Net45, Web-Net-Ext, Web-Net-Ext45, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Http-Logging, Web-Url-Auth, Web-Windows-Auth, Web-Mgmt-Console  -verbose
  • A dedicated domain service account is recommended to can grant the least required privileges. If you installed all the components on the same server you can use the local built-in server account but, if have the SQL Database/Analysis services (CUBE) deployed on a different  servers, the account is required.
  • The QoE Archiving and the CUBE needs to be already deployed.

The installation package is the same for all CQD components so, if: (a) you are installing all components you can go to step 2; (b) if you already installed the QoE Archiving and/or the CUBE on the same server, go to ‘programs and features’ and ‘change’ the package and proceed to step 2:

  1. Proceed throw the welcome screen, licence agreement, and choose the binaries install location:
  2. For this part, I will select the Portal and proceed to the configurations screen:

    Configuration options:
    sqlname-vs-instance QoE Archive SQL Server: SQL Server instance name for where the QoE Archive database is located.
    Cube Analysis Server: SQL Server Analysis Service instance name for where the cube is located.
    Repository SQL Server: SQL Server instance name where the Repository database is to be created.
    IIS App Pool User – User Name & Password: The account that the IIS application pool should execute under and access the other components. You can choose one of the local server services account, otherwise choose ‘Other’ and provide a domain service account credentials (see pre-requisites above explanation).

  3. After the validations the installation will ask to proceed until completion, hopefully without any error 🙂

Behind the CQD Portal

What happened and was configured after the previous installation steps?
This component setup  installed some specific files, created support database and made some updates on the QoE CUBE Database:
• QoERepositoryDb database was created. This database holds the portal all the configurations, customized reports, …
• ‘IIS App Pool User’ login created and assigned db_owner on the QoERepositoryDb
• ‘IIS App Pool User’ login created and assigned db_datareader on the QoEArchive database
• ‘IIS App Pool User’ added to the QoERole on the CUBE database
• IIS default web site configured with 3 folders that matches the directories and files installed.

Known ‘caveats’ regarding the installation and architecture process:

  • In rare cases, the installer fails to create the correct settings in IIS. Manual change is required to allow users to log into the CQD. If users are having trouble logging in, please follow the steps described on ‘know issues’ section of the  TechNet article.
  • Cube Sync Fails – QoEMetrics may contain some invalid records based on end user clocks. If the time skew is greater than 60 yrs, the cube import will fail. Check the Min and Max StartTime/EndTime using the selections below. Look for and delete records in the far past and very distant future, they can be disregarded and they will break up the sync processes.
    Select MIN(StartTime) FROM CqdPartitionedStreamView
    Select MAX(StartTime) FROM CqdPartitionedStreamView
    Select MIN(EndTime) FROM CqdPartitionedStreamView
    Select MAX(EndTime) FROM CqdPartitionedStreamView
  • After deploying the CQD on a new server, you can run into a problem where the Portal was not showing any data and returned a problem saying:
    We couldn’t perform the query while running it on the Cube. Use the Query Editor to modify the query and fix any issues. Also make sure that the Cube is accessible
    In order to solve it, process the CUBE object and make sure it’s accessible as described here.

How to manage and monitor the CQD Portal process

The main portal page is accessible via http://<portalserverFQDN>/CQD.
CQD-Portal-main.png

You probably will not see any data because “when the installer is done, most likely the SQL Server Agent job will be in progress, doing the initial load of the QoE data and the cube processing. Depending on the amount of data in QoE, the portal will not have data available for viewing yet.” To check on the status of the data load and cube processing, go to http://<portalserverFQDN>/CQD/#/Health.
CQD-Portal-health
Or (like my LAB) you don’t have any monitoring data to display :). After that you should see the last successful and failed update status:
CQD-Portal-health-ok

Other configurations that you can perform on the Portal are described on the Deploy CQD TechNet article:

  • Post-install tasks required to have reporting data regarding locations (buildings, networks name, subnets, BSSID)
  • By default, any authenticated user has access. This can be changed by using IIS Authorization rules to restrict to a specific.
  • Detailed log messages will be shown if debug mode is enabled. To enable debug mode, go to [CQD installed Dir]\QoEDataService\web.config, and update the following line so the value is set to True:
    <add key=”QoEDataLib.DebugMode” value=”True” />

And that’s it! you now have CQD fully deployed!
You can now see how the Lync/Skype4b is performing, and even build you own reports. Creating them is tricky, but you can learn some basics here.

<Am I missing something? maybe some more posts about it. provide me some feedback suggestions/requests 😉 >