Skype for Business 2016 client critical exploit public available

The code mentions the Skype for Business 2016 client, but the base vulnerability affects also the Lync/Skype4b 2015 client:

• Risk: severe – exposes the user data
• Exploit codes available here or here
  No user-interaction is required for the XSS to execute on the target machine. It will run regardless of whether or not they accept the message. The target only needs to be online.

Solution: apply asap the June 2017 update on the Skype4b 2016 or 2015 clients

UCaaS: Part #1 – planning to provide Skype for Business as a service?

Over the years of consulting services, managing and deploying UCC solutions, virtualization,… you might have thought several times about: why not provide them to all my customers from as a packed service?.
It’s nothing new nowadays – you called it ‘cloud’, hosting services, …

I’ve been planning and designing it for quite some years. So why not just share some memories, experience and concepts?
I decided to call it, since the 0 day, UCaaS – Unified Communications as a Service. Looks a cliché now, but it’s short and easy to catch 🙂 but also much broader than just providing Skype for Business

In this first part, I will just do a global overview that applies to any XaaS. I will focus on Lync/Skype4B on later posts.
Look at this one as a cooking lesson, starting by the main topic: the Kitchen 🙂

#1 It’s a service, so you need to see more then installing a couple of servers and connect the users! Before getting to that stage think, discuss and question about everything that comes to your mind.
Why? because we are about to run a business, any resource costs something and you need to count them on your selling price !

#2 We can go this way:
* buy a server, put on your basement or garage, connect to the internet, rent/create a webstore site and ready! (it can actually work)… or,
* everything below this line (more or less complex and as a existing company you might already have)

I like to group things to be easy to read and explain . Be aware that you might not need to own or have all on your side (you can just rent datacentre space, VM’s, backups).

Base infrastructure

The ‘hardware’, where you place it and how you reach it: Servers, Storage, Backup robots, Switching, Routing,  Firewalls, Load Balancers, Rack/Datacentre space, energy, internal and external connectivity (cabling, telecommunications, internet), …
By the way: all these also have something called ‘yearly maintenance costs’ if purchased and you will need to allocate some earnings  to replacement them when it’s time.

if planning big, consider consulting Hardware providers that support ‘pay as you grow model’. Many of them have cloud-ready solutions from small footprint up to large scale (and you can find some nice surprises on less-known brands)

Support infrastructure

Some invisible, but ‘must have’ systems:

• Virtualization- of course you will use it 🙂 and on this one we can even have mixed scenarios as the group above (virtual firewall, load balancers,…)
• Monitoring – Ever heard of an SLA ? if you are providing a service you will have to agree on an uptime. How can you measure and show to the customer? How can you detect failures or when you need more resources?
• Backup and DR – Are you ready to loose your data? what about the customers data?
• Security – you will need to manage patching and upgrades, antivirus, IDS, IPS… it’s a dangerous world outside, waiting to steal your customer data or take down you business.

 

Customer interfaces

This is your front porch and you should not hide it. If the prospective customers don’t like it, will they trust the inside of the housing?

• Customer infrastructure connectivity – the way that the customer systems and users will connect to your services. The simplest way is the internet, but it would require for some services, WAN, private networks and interoperability/integration solutions.
• The customer portal/tools – This can be from a simple status/account/billing view up to a self-provisioning, self-management
• Ticket / support handling – five customers might be easy to deal with phone calls and emails, but what about 30,100,…?

Software

I include on this section separately, because not everybody is aware of some legal aspects. Let’s take a look at Microsoft products: you cannot just buy a Windows license, install on your server and charge it to one (or more) customer(s) for a running service there.
Microsoft is clear on this: if you are a Hosting/Service Provider, you need to buy licensing throw a SPLA .
Like Microsoft, VMware and many other vendors provide (or enforce) this model and is not a bad option:

• The advantage is that it allows you to pay monthly for what you really use –
  this is the pay-as-you-use model
• The inconvenient is that you need to report the usage periodically and allow auditing to your business

My advice is:

• as a Service Provider, contact the vendor and explain your intentions. They will help you to find the most profitable solution… most of the times.
•  Don’t try to find ‘loopholes’…trust me: it will cost you much more later!

Shared services

Here you put all the platforms and services that can be shared between multiple.
Great examples are web servers hosting multiple websites. But pushing your skills to the limit, you can have a lot more. Multitenant solutions also would fit on this group.

Dedicated your best resources planning them! This is where your cost savings make a difference.

Dedicated services

This is what your best customers are looking and willing to pay for.
It’s your cash cow – the more you have here, the quicker your revenue increases. These business models have more opportunities, specially combined with standardized offers.

It has every group of systems serving unique customers:

• It should be a ‘block model’ – same deployment and standardized procedures, automation and self service tools, will keep operational costs low and predictable.
• But you can also include very ‘$pecific $olutions’ – these one gives you the opportunity to upsell consulting and managed services.

Skype for Business services model would fit on this group… but it might also fit for the group above? 😉

Not there yet

If you already own a company then this is known to you: Work office place, furniture, energy, personal computers, HR, billing / account management, mobile devices, transportation/gas… you also pay for that, right

All done! What now?

After joining and calculating all the pieces, test yourself doing at least these questions:

• What is the cost per service/per user? How much a VM with a specific size costs?
  This is the lowest value will charge for your service.
  => Capitalism rule: to earn money you need to sell it for more than you pay for
• What is the break even point?
  simulate, over and over the time! Be ready to answer: how many customers (or users) do you need to have to cover all the costs made so far?
  Whomever is going to put money on this will make do that question before writing the checks (even yourself if you got the money)
• What is the ROI?
  Investors, banks or stake holders will look for this (and many other tools).
  If you need financing you need to promise when and how much will you pay back.
• Psychological one: are you an entrepreneur?
  It needs investment analysis, financial control, technical know-how, purchasing/selling/negotiation skills, HR management, and so on. As the business creator you need deal with of them.
  This will not be a one-man-show for long. Sooner or later you need help trust and delegate to others. Starting with someone it will look less difficult.

This might seem directed to start-ups guys, but it applies to existing companies. Product and/or Business developers need also to assume the above challenges, the problems, assume risks and responsibilities.

You are putting you neck and reputation on it and get ready to accept failure.
But of you planed all (not just a suicidal gambler), showed you the balls, and if successfully, the personal reward is priceless.

It’s still not over! there are some more strategic decisions for a go-to-market, on part 2 (writing in progress).

 

The minimal amount of servers for a full HA Enterprise Pool: less than 4

By the end of 2015, I started enumerating some challenges that the Lync/Skype topology presents when you just need minimal resources and provided some answers. Since the beginning of Microsoft UC and the more I dig inside the product versions, I came with some out-of-the-box personal challenges:
– 4/Feb/2012 – Installed a Standard Edition Lync 2010 in a Domain controller (DNS, CA) – LyncIn1Box
– 11/Dec/2016 – Installed Skype for Business 2015 in a Windows Server Core – The smallest Skype for Business front-end server

As strange as it sounds, there are some small companies around the world that depend on Lync/Skype for critical business mission but don’t have enough resources. Companies with small amount of users can go for Office365, but it might no cover all the features of the on-premises. Pool pairing (1 STD Edition + 1 STD edition) might also not be possible if there is no feasible secondary location.

A short technical review of what roles you need to cover for High Availability (HA):
* Front End Lync/Skype server roles
* Back-end databases  – you need a SQL mirror or a Cluster/Always-On
* File Share – can be provided by a DFS or a File Share cluster

Now we can start asking: how many resources do I need to deploy the above HA ?
Here some possible answers. Note that ‘something’ means that a SQL HA needs 2 Instances and a sort of a witness (either a SQL server or a File Share/disk quorum)

Microsoft recommended: 5.x (5 servers + something)

Following Microsoft recommended practices, you will need 3 front-end servers and a redundant back-end database. File Share can be easily deployed on the Back-end servers.

Microsoft minimum: 4.x

As Microsoft allow the usage of two Front end server, but: “This small pool will not provide a robust high-availability solution like a larger pool would, and needs extra care in managing“.

My recommended PoC: 3.0

By joining all the roles across 3 windows servers, you can have one server down and the built-in automatic failover mechanisms will take care of that.

My minimum PoC : 2.x (2 servers + something)

Redundant means at least two systems, so this is the absolute minimum :). It has the same risks as Microsoft states for two servers.

The main reason is the fabric quorum: If you unexpectedly loose a server, the fabric cannot elect the owner of the several roles… even if there is only that server. There will be the need of a ‘soft’ quorum reset (manually or a triggered task)
But if you shutdown a server/services gracefully all services will be transferred to the working node.

About the 2.x equation:
* If you use a SQL mirror, a SQL witness is required (which actually means an additional/existing Windows server)
* If you use SQL clustering/Always On, you will need: (1) a file share witness -doesn’t have to be a Windows but can count as a server :)- (2) a quorum disk (share storage) – and this is the ‘true’ 2.x–

Final thoughts

Deploying this ‘minimalist’ solution is more about ‘out-of-the-box-thinking’ than tweaking (actually it’s only one). The normal Skype for Business setup wizard will install normally all the roles defined on the topology.

Notes about this ‘minimal counting’:
* It’s about adding Skype services: it assumes that you have a Windows Domain, a CA and a sort of Load Balancer.
* SQL Clustering/AlwaysOn is still on test stage, but the service failover should behave similar as the Mirroring.
* If you already have a SQL HA in place at the datacentre, you can reuse it. This will make MY and MS minimal requirements equal to 2.
* At this time I excluded the Edge, Office Web App roles or Persistent Chat. The first one will required more resources, the second role… is for an updated post 😉

Logging scenarios that crash and cripple the RTCCLSAGT

Another quest with an issue that appears to be there since some time of Skype for Business 2015 server (if not even Lync server. I decided to find out and document it.

ISSUE

Consider the following scenario:
1) You start the Skype for Business logging tool
2) Pick one of the following builtin scenarios (CmdletDebug or IISLog)
3) As soon as you start this scenarios on the selected servers you will notice the following error message on the logging tool output window: “ResponseMessage: Error code – 20000, Message – Unknown error – Error calling agent <FE-FQDN>; Could not connect to net.tcp://<FE-FQDN>:50002/. The connection attempt lasted for a time span of 00:00:02.0071248. TCP error code 10061: No connection could be made because the target machine actively refused it 10.101.128.20:50002. . Please refer CLS logs for details.”

By this time you will notice that the (Skype for Business Server Centralized Logging Service Agent) RTCCLSAGT service has crashed on all the servers you initiated the trace, with the following event error id 33040:
“Centralized Logging Service Agent Error starting background thread to process traces.
Log Type – IISLogManager, Error – Object reference not set to an instance of an object.
Cause: Internal error
Resolution: Examine error details to determine resolution.“

Worst, the service will crash every time you try to start it with the same error. You find yourself without any logging capability on your servers!

CAUSE

The reason is that the two scenario contain invalid trace components called ‘Internal’ and ‘External’. Somehow this triggers an internal failure on the IISLogManager component and damaged it.
When you edit those scenarios, you will notice a warning about the components

SOLUTION(s)

This is how you can fix the two problems:

• Repair the failed service start (no server downtime required)
  Run the repair of the Skype for Business 2015, Core Components.
  
  Pay attention that two services need to be set to Automatic (Delayed Start) or it may fail on Windows 2012 Servers (as the above picture). Just noticed the instructions on my previous post.
• Fix the scenarios to prevent from happening again:
  1) notice the valid components and flags.
  2) delete the scenario
  3) create a scenario with the same name and all the components (excluding the ‘External’ and ‘Internal’)

Cannot repair/reinstall Skype for Business core components on Windows 2012 Server

There are some specific situations where you need to repair or install an a new component of Skype for Business 2015 server. Because of a corruption, I only faced this very recently, but it’s there since November 2016.

ISSUE

You have the following conditions on your Skype for Business server:
– Windows 2012 Server with .Net Framework 4.5.1 installed (with or without the most recent updates)
– Skype for Business 2015 Nov/2016 cumulative update (or later) installed

You are unable to run the repair option of Skype for Business 2015, Core Components with the error message: ‘…installation requires Microsoft .Net Framework 4.5. Installation cannot continue.’

CAUSE

The Nov/2016 cumulative update added a new ‘Launch Condition’ to the .msi installation:
– before it validates the existence of .Net Framework 4.5
PATCH Or MSIPATCHREMOVE Or REMOVE Or NETFRAMEWORK45 OR SKIP_NETFRAMEWORK_CHECK
– now, it checks for the existence of .Net Framework 4.5.2 (releasecode 379893)
PATCH Or MSIPATCHREMOVE Or REMOVE Or (NETFRAMEWORK45 >= “#379893”) OR SKIP_NETFRAMEWORK_CHECK

SOLUTIONS

I’ve identified 3 possible solutions:

A) The ‘SysAdmin Pro’
1. Manually run the following command line (with the skip .Net check):
msiexec /i {DE39F60A-D57F-48F5-A2BD-8BA3FE794E1F} SKIP_NETFRAMEWORK_CHECK=1
2. Follow the setup GUI and choose the repair option
Note: the msiexec repair option ignores the custom parameters and will give the error

B) The ‘sneaky sysadmin’
1. Take ownership (and grant write permissions) of the registry key
HKLM\SOFTWARE\Wow6432Node\Microsoft\NET Framework Setup\NDP\v4\Full
2. Change the REG_DWORD key ‘Release’ to a value equal or greater than 379893

3. Run the repair option
4. Change the REG_DWORD key ‘Release’ to the original value

C) The ‘let’s upgrade and see’ sysadmin’
1. Update .Net Framework to version 4.5.2
2. Run the repair option
NOTE: This was not tested, but it’s recommended at least to rerun repair option on all Skype Web-related components. The main advantage is that is a permanent solution… pr at least until MS release another CU requiring 4.6 😉

Final notes and caveats

I could not find the reason for this change documented:
* Skype for Business per-requisites still points to .Net Framework 4.5
  (Skype for Business runs fine on a Windows 2012 server with .Net Framework 4.5)
* The recent cumulative updates don’t push/required an update to .Net Framework 4.5.2

Important: After you repair the Skype for Business 2015 Core components, it disables the two associated windows services:
* RTCCLSAGT (Skype for Business Server Centralized Logging Service Agent)
* REPLICA (Skype for Business Server Replica Replicator Agent)
Just make sure that you set the services back to Automatic (Delayed Start)

Warning: May2017 windows update break Lync 2010,2013 and Skype for Business 2015 services

This is a quick post notice since I’m still trying to analyse the cause, and more Engineers are reporting the same issue.

ISSUE

 After you install the latest Windows May2017 updates, Skype for Business 2015 will start reporting the following errors:

• External Users reported that couldn’t use WhiteBoard, Polls, Q&A or present PowerPoint with the following errors messages:
  We can’t connect to the server for sharing right now.
  Network issues are keeping you from sharing notes and presenting whiteboards, polls and uploaded PowerPoint files.
  
  

  

   

• Front-end servers: (Event ID 41026/41025) “No connectivity with any of Web Conferencing Edge Server, External Skype for Business clients cannot use Web Conferencing modality”
• Edge servers : “Web Conferencing Server connection failed to establishOver the past 3 minutes Skype for Business Server has experienced incoming TLS connection failures 1 times(s). The error code of the last failure is 0x80072746”
  or event id 41026
  

Cause

The issue seems to be some secure hardening on the TLS negotiation.
According to MS Support, it looks like an ‘update blooper’ on the May2017 patch:
“This update adds an additional check on Enhanced Key Usage (EKU), since all Lync/ SfB Server usually use the Web Server template they will only have the Server Authentication in the EKU.“

Impact and affected systems

Impact level: low. Web conferencing service (whiteboard, powerpoint presentations ).

 According to MS: “The issue has been reproduced on Lync 2010, Lync 2013 and Skype for Business 2015 on all supported server versions (2008r2, 2012, 2012r2).”

Workarounds

– uninstall the May 2017 Security and Quality rollup for the .Net Framework 4.5.2
– Request new Edge Internal certificate with the Client and Server Authentication
– On the Front Ends disable the check for the Web Conferencing Service (follow this post for details on how to do it)

Update 18th May 9:30 –  Update content based on this post (thank you Erdal for pointing me to that blog)

Update 24th May 6:30 – Updated with end-user error messages and picture. Microsoft published KB4023993 describing the issue and the workarounds described above

Skype4B interoperability: Cisco Expressway

Cisco Expressway “offer users outside your firewall simple, highly secure access to all collaboration workloads, including video, voice, content, IM, and presence. Collaborate with people who are on third-party systems and endpoints or in other companies. Help teleworkers and Cisco Jabber mobile users work more effectively on their device of choice. Cisco Expressway allows you to do all this and more.” (source)

Using Skype for Business terminology, this is the “Edge Server” of Cisco. But it besides the described functions, it has one that allows you the integrate Cisco Video endpoints (desktop client, video conference rooms up to telepresence system) with Lync/Skype for Business clients that allows SIP dialling from both sides with audio and HD video: search for the contact and press the call button.

Skype user view

Cisco meeting room view

 

The infrastructure configuration is based on a ‘dedicated expressway’ configured as a Skype for Business ‘trusted application’. The rest is all about SIP domain ‘static routing’.

Expressway topology for on-premises Skype for Business and Cisco video endpoints 

Great features of this integration:

• IM & presence is possible between systems;
• Desktop sharing is supported;
• Both Cisco and Microsoft can share a single SIP domain;
• Skype for Business remote clients can use the Expressway on the DMZ (TURN server) for audio traffic with Cisco endpoints;
• Skype for Business clients can make and receive audio/video calls with external parties whom the Cisco has federation with.

There are also some limitations:

• Presence is based on SIP simple. Only can have the status ‘Available’ and ‘Busy’;
• Skype for Business clients can join a multipoint conference in Cisco, but Cisco endpoints cannot join Skype meetings.

 

In conclusion, although it’s not a better solution than Polycom, the Expressway leverages the existing investment  already done on Cisco infrastructure and can be extended to a complete 3rd interoperable federation solution: