IT based Communications

a different Unified Communications site

You can’t record after you install MS15-097, or… can you?

Microsoft just added the following known issue regarding the latest September 2015 Security Update (SU):

  • You install security update 3085500 that is documented in security bulletin MS15-097 Security Update for Lync 2013 (Skype for Business) that was released on September 8, 2015.
  • You join a meeting by using the Skype for Business client (version: 15.0.4753.1000).
  • You click the More Options button, and then you click Start Recording.

In this scenario, nothing happens. You don’t see any notification or visual indication that recording has started. Additionally, when you open the recording manager, you don’t see any file or progress status.

This problem is scheduled to be fixed in the October 2015 update for Lync 2013 (Skype for Business).”

I try to check for this, but on the several clients, in different update levels (ex: update directly from April 2014 CU), I can still successfully record meetings.

There is no more explanation on the KB3099414, but there might be some pre-conditions to be meet before you to experience this issue. You might still want to apply the Set2015 SU (there are some of important fixes) and test if you’re enterprise users are affected.

The only reference so far is on a post in the Office365 community, where Microsoft Support confirms the issue.

Lync Server 2013: install (or not?) July 2015 CU before installing September 15 SU

These Microsoft KB documents where updated on the last 24 hours, got me puzzled:

KB3080353 (25/09/2015): Description of the security update for Microsoft Lync Server 2013 (Web Components Server): September 8, 2015
To install this security update, you have to install the latest July 2015 update Lync Server 2013.

KB3098577 (24/09/2015): Sign-in and query errors after you install MS15-104 Security update for Microsoft Lync Server 2013 (Web Components Server)
After you install security update 3080353 …, you experience one or more of the following symptoms in Microsoft Lync Server 2013:

  • Users can’t sign in to your dial-in page.
  • Lync Mobile clients can’t sign in.
  • External clients can’t sign in.
  • Address book web queries fail.
  • Users are prompted for credentials for some web services after they sign in internally to Lync desktop clients.”

But below on the KB text:

“If you install Lync security updates by using the Cumulative Server Update Installer (LyncServerUpdateInstaller.exe), the installer should prompt you to install the correct prerequisites
Security update 3080503 (MS15-104: Security update for Microsoft Lync Server 2013 (Web Components Server): September 8) can be installed without previously installing the required July 2015 cumulative update 5.0.8308.920 for Lync Server 2013.”. A new version of LyncServerUpdateInstaller.exe is shipped together with the security update.

My concerns are:

  • The issue is applying the September “WebComponents.msp” (you can find it separately on the download details) on the download patch without updating the one from July (ex: using windows update)
  • The “LyncServerUpdateInstaller.exe” is basically a package of the most recent released .msp files for each components
  • So, what WebComponents.msp version is inside the September LyncServerUpdateInstaller ?

Just for the safe side (since I have a large deployment with 4 pools and 28 Lync Servers), I will apply the July 2013 LyncServerUpdateInstaller and after that the September one.


Time to compare to the July and September 2015 updates. It all comes to the WebComponents.msp content. Beyond the ~1500 files with some date differences, we have this:

Filename Date July 2015 CU September 2015 SU
WebComponents.msp comparison 1511 Files total 1507 Files total
[Lync_Install_Dir]\Web Components\LWA\Ext\SWF\

[Lync_Install_Dir]\Web Components\LWA\Int\SWF\

expressInstall.swf 29.04.2015 included in .msp Not included in .msp
NAPImessage.swf 29.04.2015 included in .msp Not included in .msp
Swfobject.js 29.04.2015 included in .msp Not included in .msp
[Lync_Install_Dir]\Web Components\Web Scheduler\Ext\Scripts
jquery.js 21.07.2015 Not included in .msp included in .msp

These file are part of the puzzle, especially if they are not part of the first release of Lync. They ‘swf’ were introduced on the December 2014 CU, and update on May 2015 CU and July 2015 CU (check the file information section on each KB). This means that applying the Lync servers with September 2015 SU without the  May 2015 CU, you will have any old version or even none (if you didn’t applied nothing before December 2014)

KB3055014 and KB3085500 introduces an unknown xml “400 Bad Request” issue

I noticed recently on several client troubleshooting (using the Snooper tool) on a particular “400 Bad Request” message that appears on the end calls.
After several cumulative updates (CU) history regression testing, I found that this behaviour started appearing with the latest two CU (or correctly, the security updates):

  • KB3055014 – security update for Microsoft Lync 2013 (Skype for Business): August 11, 2015
  • KB3085500 – security update for Microsoft Lync 2013 (Skype for Business): September 8, 2015

If you have any of those updates, you will find the 400 error message on every call (PSTN or Skype) when the called party disconnects. Here’s an example of the same call I made to a PSTN number:

  • Until the July 2015 update, the client will send an “error” report in xml format for the front-end server
  • After the August/September 2015 updates the same report is now different (similar response as a Skype-to-Skype)
    There’s a <progressReport/> ending without any initial <progressReport>
  • The server detects this incorrect xml format and that’s why you see the “400 Bad request” and a “Client.BadCall” fault

For the end-user there is no visible or functional impact.
I still don’t know if there are any impacts for the backend services. The call diagnostics for CDR and QoE diagnostics are correctly sent on another SIP message.

iOS9 breaks lync and s4b meetings URL

iOS9-noOpenmeetingURLThere is one more issue when you upgrade your iPhone or iPad to iOS9: Looks like when try to open a meeting url on Safari, the Lync client is not launched and you stay on the meeting webpage (like the picture on the right).

Looking at the behaviour and some initials traces, looks like there’s some update on the Safari web browser that is not allowing the Lync javascript https://<meetingurl>/JavaScript/Launch.js?lcs_w15_onprem5.0.8308.726 from lauching the client.


  • workaround: install/use  the Opera mini web browser on the iPhone/iPad
  • Microsoft confirms the issue on KB3097592 and has another workaround.
  • 25.09.2015 – iOS 9.0.1 update doesn’t resolve the issue
  • Microsoft just released Set2015 CU13 (don’t get confused with Set2015 SU) that contains a web component update to address the issue (web pages and scripts) were updated and safari can now detect and launch the Lync client.

Lync issue: The mistery of an iPhone, a HLB and a the missing cookie

You might not face this particular issue, but it could be helpful to troubleshoot similar on how the mobile clients communicate with the external web services.
Everything was running normal on this customer, until the day we plan to implement a two factor authentication (TFA) engine. To understand the nature of the issue here’s the background design:

  • In a standard deployment the ‘reverse proxy role’ will forward the external web services [public IP]:443 to the [front end pool]:4443
  • The TFA solution replaces this roles and ‘intercepts’ the web services data (the components are installed on the Edge servers, work separately for mobile client devices). Besides DoS protection it validates user/device pairing
  • For a high availability solution, hardware load balancers are used. Since these are web services, cookie passive affinity is used (a HLB cookie is inserted if the server response adds is own cookie). We could use MS recommendation of source IP affinity, but since there’s a lot of locations (with internet access through NAT) with thousands of users we want to ensure an equal load distribution for the TFA services.

Everything looks good and ready to go, but then this interesting behaviour start occurring:

  • Windows and Android clients worked fine;
  • but iOS (iPhone and iPad) start fail to sign-in;
  • bypassing the ‘TFA filter’ will made all devices connect okM
  • configuring the HLB to send all traffic to one node only, will also made iOS clients sign-inM
  • With HLB Source IP affinity, all worked fine;
  • Cookie insert affinity will still fails for iPhones and iPads;
  • The latest Lync server and HLB firmware updates would not solve the issue.

You (and at the beginning I) would assume that the problem is somewhere related to the HLB on the TFA, right?
Well… troubleshooting the iPhone client give an additional surprise:

  1. When the client authentication process starts, the TFA component returns the server response to the client adding his cookie. The HLB will then also includes is cookie so the remain authentication process and traffic will be sent to the same server.
  2. but then I just notice the next client POST request is ignoring the HLB cookie and is only sending the TFA cookie !

The reason of the failure is that, without the HLB cookie, all client traffic requests are distributed across the two TFA and the other Front-end servers that didn’t participate on the initial negotiation and… the sign-in process fails.Looking at the same process for the Windows (Skype for Business) and Android (Lync 2013), all the cookies are sent.

After several reconfigurations attempts and troubleshooting, the issue was solved by defining on the HLB the cookie path (‘/’ instead of empty) , and the iPhone client start acting as it should:

  1. This time the HLB cookie is appending the path
  2. And the iPhone client send all the cookies on the next POST request

So at the end, it looks like that there is some sort of undocumented behaviour on Lync 2013 client for iOS.
Possibly it will disappear on the next update: Skype for Business… or not ?:)

Lync Mobile 2013 with iOS 9 Sign-In fails

IiOS9culturef you just upgrade your iPhone/iPad to the iOS 9 released on the 16.09.2015, and have a language different from a region where the language is not native, like me (left picture):

iOS9-lyncfailYou might be unable now to login with your Lync 2013 client with a strange error message (right picture):

Doing some “sniffing”, looks like Lync is sending a Culture ID that the server doesn’t recognize and replies with a “401 Bad request” in my case en-CH doesn’t exist:LyncIphone-cultureID_issueOne workaround do far seems to be: define a language that exists on the region with so Lync can send a valid culture ID.

Tkmikal found this Microsoft KB3096704 recognizing the issue:
“This problem is fixed in the Microsoft Skype for Business for iOS app that will replace Lync for iPhone and Lync for iPad when it’s released. No fix for this issue is scheduled for the current releases of Lync for iPhone and Lync for iPad”.

UPDATE (21.09.2015):

This should stay by here, but then I went “under” the iOS and found an additional cause. If you have the location services disabled during the iOS process update, it will not be able to set all the new information parameters about language and region for version 9.

So, here’s a stable solution:

  1. Turn on Location Services on the iOS device, and be sure to have an internet connection active;
  2. Switch the language to another one and back to the one you want – this is a fast way to make the iOS download all the iOS9 configuration format;
  3. Start Lync and you are now able to sign-in;
  4. You can now disable back the location services. Lync will still sign-in correctly because iOS9 language settings are now installed.

Skype for Bussiness glitch: the flashing call monitor on PSTN inbound call

When you have a 12’000 users infrastructure, you are sure that the any Lync client issue will be detected.

On this case someone notice a glitch with the “Skype for Business skin” only. After some testing we pinpoint the issue:

  • You receive an PSTN call;
  • When you start using another desktop application the call monitor will switch to the miniature version and…it flashes from time to time!

The issue was reported to Microsoft 2 weeks ago.

Although is not documented or confirmed by technical support, it seems that the security update for Microsoft Lync 2013 (Skype for Business): September 8, 2015 solved the issue.

Lync (serious) vulnerable and exploited without MS15-034

The recent MS15-034 security update addresses a vulnerability on how the Windows HTTP stack (http.sys) handles requests.
Although you affected Operating System component is related to IIS, there are many applications that can rely on Windows HTTP stack. And Lync is one of them !
How serious is this vulnerability and why you should patch immediately?

  • It can be used for DoS attacks, but there’s a chance to be used to run code remotly
  • Any user can run an exploit of some type without any special permissions and good knowledge;
    Can be just a simple copy > paste code (see PoC)
  • Your Lync front-end servers can be exploited by an internet attacker, if the reverse proxy role (and/or the firewall) cannot detect and intercept the exploit attempts.

What Lync ‘roles’ are affected?:

  • Front-End – There’s a lot of applications/pools that can be exploited
  • Edge server – not affected from the outside. But the internal DMZ replica service (typically 4443) can be exploited
  • Persistent chat – not affected
  • (SQL) Monitoring reports – affected
  • Office Web Apps – affected

To show you how easy the exploit can be built and run, here’s a simple Proof of concept. I just needed 10 minutes to find a possible http request and run cURL on an internal PC without any admin rights:

exploiting-pcThe server running Lync will stop responding and (if you are fast enough) you will see the operation system generating a dump report, before restarting.


An exploited server will also display a MER message when you logon to it:


You might want to look carefully for Lync and other collocated applications that can also allow an exploit. This command can be used to determine what is relying on http:
netsh http show servicestate | find “://”

So it’s better to start patching all windows operating system on you network… fast

Additional references:

Lync 2013 Cmdlets cheat list

I don’t know about you, but I have some difficult to memorize powershell commands.
In Lync 2013 there are 754 cmdlets available: some that I know and use on a daily basis, a few are specific for Lync Online, several that I never used or need at all but from time to time I find out some that are useful for specific tasks and that… I forget them later.

Cmdlets-sampleGoogle is still my first tool to find out how to get some particular job done in Lync, but I decided to create a summary list Microsoft TechNet Lync 2013 cmdlets (see picture sample on the right) with:

  • The cmdlet that includes the url to the specific TechNet documentation
  • The short description of the command as is on TechNet
    (took me 2 days to build a function to extract this part of text using the page url of the first column.)
  • Add-on: default Lync RBAC groups and command assignment
    It was also another need for me to allow me to plan RBAC permissions grant and create specific roles

If this could be also useful to you, the file can be downloaded here: LyncCmdlets or at Technet

On my next available time, I plan to add more features (like categories)

Lync client update KB2910927 (and later) breaks embeded urls

LyncKB2910927bugI just discovered an issue when installing the December 9, 2014 cumulative update (CU) for Lync client 2013 (KB2910927)

When you try send an url with the prefix www you get the error message “the web page … was not loaded in response to you clicking the link because it is either invalid or restricted for security reasons” (picture on the right). This will appear even if have no URL policies restrictions.

– prepend http:// to the address you want to send, or
– uninstall KB2910927. The previous CU doesn’t have the issue.

Googling on the Internet I also found that a similar behaviour has already been reported and confirmed to Microsoft, and published as KB3053114 issue.

UPDATE: The latest Lync update (KB3039779) fixes the issue.

Lync issue: export-csuserdata failed with “Data in NULL”

It started when a backup script that I implemented logged a failure on the export-csuserdata process. Running the  powershell command isolated would generate the same error text: “Data is Null. This method or property cannot be called on Null values.”
export-userdataerrorSo, we have a sort of user data corruption and not enough clues to locate which of the 4’000 accounts was. The export-csuserdata generated an incomplete zip file but comparing with the last complete backup wasn’t enough to compare and find out what account was generating the error. With some brainstorming and scripting I finally identified the user, and using the export with the userfilter parameter generate the error.

Time to reverse-engineering the root cause:

Step #1: A network and then a SQL trace on the Backend database, show that the export-csuserdata calls a store procedure of the rtcxds named ‘XdsBackupAllItems’ with the user’s SID on the parameter

Step #2: Executing the store procedure confirmed the output data with two NULL values

SQLCMD -S Server249\LYNCCORE -d rtcxds -Q "exec dbo.XdsBackupAllItems @_TenantId='92e2a9d7-41d7-4c43-bbba-46122f5d5ab6', @_ExtensionParameter=null"
 <app:DocItem xmlns:app="urn:schema:Microsoft.Rtc.Management.Xds.AppLayer.2008" Name="urn:lcd:M****_K*****@s******.com" ItemId="92E2A9D7-41D7-4C43-BBBA-46122F5D5AB6" Owner="04DDEDCE-ED20-514D-8586-58BEE1904EE8" OwnerPool="pool0101.s*****.com" Signatu 

 <app:DocItem xmlns:app="urn:schema:Microsoft.Rtc.Management.Xds.AppLayer.2008" Name="urn:confs:M****_K*****@s******.com" ItemId="92E2A9D7-41D7-4C43-BBBA-46122F5D5AB6" Owner="04DDEDCE-ED20-514D-8586-58BEE1904EE8" OwnerPool="pool0101.s*****.com" Signatu 

 <app:DocItem xmlns:app="urn:schema:Microsoft.Rtc.Management.Xds.AppLayer.2008" Name="urn:upc:M****_K*****@s******.com" ItemId="92E2A9D7-41D7-4C43-BBBA-46122F5D5AB6" Owner="04DDEDCE-ED20-514D-8586-58BEE1904EE8" OwnerPool="pool0101.s*****.com" Signatu 
 <app:DocItem xmlns:app="urn:schema:Microsoft.Rtc.Management.Xds.AppLayer.2008" Name="urn:hcd:M****_K*****@s******.com" ItemId="92E2A9D7-41D7-4C43-BBBA-46122F5D5AB6" Owner="04DDEDCE-ED20-514D-8586-58BEE1904EE8" OwnerPool="pool0101.s*****.com" Signatu  

Step #3: Manipulating the store procedure we can identify which field(s) is generating the NULL

Step #4: After several attempts stripping down the SQL statement, I found the two records on the table ‘Item’ that didn’t have a reference to any DocId field on the ‘Document’ table.

I don’t know what caused the ‘orphaned’ relationship, since there are tables field constrains to prevent that…. but it’s time to find a solution that doesn’t compromise all existing data.

Simulating on a Lab environment showed that moving the user between pools or even disabling from Lync will not clear the bad records, and once the user was back on the pool database the issue will return.

The fix turn out to be more simple. Since the two records on the ‘Item’ table don’t have the constrain relation with the ‘Record’ table, they could be deleted without problems with the following SQL statements:

DELETE FROM [rtcxds].[dbo].[Item] WHERE ItemId = ’92e2a9d7-41d7-4c43-bbba-46122f5d5ab6′ and DocId = ‘17941959’
DELETE FROM [rtcxds].[dbo].[Item] WHERE ItemId = ’92e2a9d7-41d7-4c43-bbba-46122f5d5ab6′ and DocId = ‘17941967’


Lync bug: the disappearing phone numbers

It started with an user reporting that, when he changes from networks (ex: wired to wireless) his phone numbers on the Lync client disappear. After more than 48 hours of simulation I was able to identify the issue. It turns out that when you have a Distribution Group on your contact list where your account is member, when you expand the list, your users Active Directory phone numbers disappear. Here’s the issue report:

The user personal phone numbers on the Lync client disappear when he expands an Active Directory distribution list where he is a member.

When the phone numbers disappear, they become unavailable to several call features (transfer calls, user the ‘join audio from’ in meetings)


  • Affects all telephones numbers of the user stored on Active Directory.
  • Affects users that have (active directory) distribution groups where their user is a member
  • Doesn’t affect phone number entered manually by the user on the Lync Client

Steps to reproduce
1. Check in your Lync client that the numbers are visible: ‘Tools > Options > phones’
Lync numbers OK

2. Search and add an active directory distribution list where the user is a member;

3. Expand the distribution list (if Lync does’t automatically do it) , go again to ‘Tools > Options > phones’ and you see that that some or all numbers disappeared
Lync numbers gone

The user does not need to logout from Lync. Search for his own contact name and the AD numbers will appear again.
Lync numbers back

Additional notes

  • After using the workaround, collapsing or expanding the same distribution list, the phone numbers don’t disappear until you logout/login back again to the Lync client and expand a list;
  • If you have more similar lists on the personal contacts and expand it will cause the same issue;
  • If you close the Lync client with the Distribution Group expanded, when you sign-in the number will disappear;
  • Even with the Distribution Group collapsed the number migth disappear at the sign-in.

UPDATE: the issue has been solved with the September 8, 2015 security update (KB3085500)

Lync issue: “We could’t reach …”

wecouldtreachDuring the last weeks of a very large deployment project that I’m working, I faced  the following different problems but with a similar pattern.
This customer’s Lync infrastructure has 2 sip domains (let’s just call and and the issue was occurring with users from different domains:

  • tried to make (was delegated permissions to) call on behalf of
  • fail to join audio PSTN (Call me at number…) if the conference organizer was
  • User agents fail to make PSTN calls on behalf of an response group with a different SIP Domain (this one referenced in a post by Joachim Dissing)

In all cases the user received the same error message “We couldn’t reach…”.
A look on the client Lync-Uccapi-0.uccapilog file revealed an INFO message with a surprising message
callmeissue-uccapilog“User does not exist”, source = >edge server> (?!).
Time to trace the outboundrouting on the Front End Servers, which reveal also something peculiar:

TL_VERBOSE... (OutboundRouting,OutboundRoutingTransaction.constructor:outboundroutingtransaction.cs(382))Creating a OutboundRoutingTransaction object (3)
TL_VERBOSE... (OutboundRouting,OutboundRouting.OnRequest:outboundrouting.cs(289))[3719388760]Enter
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.ProcessIncomingRequestHeaders:outboundroutingdispatcher.cs(1520))[3719388760]Enter
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.ProcessIncomingRequestHeaders:outboundroutingdispatcher.cs(1552))[3719388760]Referred-by header found: <>;ms-identity="MIIBwgYJKoZIhvc....hg==:Wed, 24 Sep 2014 12:24:33 GMT";ms-identity-info=";;transport=Tls";ms-identity-alg=rsa-sha1
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.ProcessIncomingRequestHeaders:outboundroutingdispatcher.cs(1678))[3719388760]Exit
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(242))[3719388760]From uri:
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(243))[3719388760]From User Uc Enabled: True
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(246))[3719388760]ReferrerUri:
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(247))[3719388760]Referrer Uc Enabled: True
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(248))[3719388760]Referrer is Local: True
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(249))[3719388760]Referrer is inCurrentDeployment: False
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(250))[3719388760]Referrer DeploymentLocator: NULL
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(256))[3719388760]IsAvMCUDialOut: True
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.OnRequest:outboundroutingdispatcher.cs(257))[3719388760]Alternate Tel URI: tel:+80001
TL_VERBOSE... (OutboundRouting,EmergencyCallHelper.IsEmergencyCall:emergencycallhelper.cs(38))IsEmergencyCall = False
TL_NOISE... (OutboundRouting,Settings.GetMatchingVacantNumberEntry:settings.cs(323))Checking for Vacant number entries for +90000
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.CheckAndRouteVacantNumberRange:outboundroutingdispatcher.cs(1838))[3719388760]+90000 does not match any Vacant Number range
TL_NOISE... (OutboundRouting,Settings.GetMatchingCPSEntry:settings.cs(291))Checking for CPS entry for +90000
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.CheckAndRouteCallParkService:outboundroutingdispatcher.cs(1165))[3719388760]+90000 does not match any CPS range
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.HasAvoidPSTNTollByPassPolicy:outboundroutingdispatcher.cs(2577))[3719388760]AvoidTollByPassUsage not found.
TL_VERBOSE... (OutboundRouting,OutboundRoutingDispatcher.IsAnonymousUser:outboundroutingdispatcher.cs(3221))[3719388760]user URI is not anonymous
TL_VERBOSE... (OutboundRouting,RoutingHelper.AddHeaderIfNotAlreadyPresent:routinghelper.cs(169))[3719388760]Adding ms-conference header with value true
TL_VERBOSE... (OutboundRouting,RoutingHelper.AddHeaderIfNotAlreadyPresent:routinghelper.cs(185))[3719388760]Adding MSReferrerPhone header with value <;user=phone>
TL_INFO(TF_PROTOCOL) (OutboundRouting,OutboundRoutingDispatcher.ProcessOutboundRequestToPstn:outboundroutingdispatcher.cs(1401))[3719388760]Target Domain and caller domain not same, sending request on its way...
TL_VERBOSE... (OutboundRouting,ClusterPingEngine.TimerCallback:clusterpingengine.cs(221))(0000000000D28422)Enter TimerCallback
TL_VERBOSE... (OutboundRouting,ClusterPingEngine.GetQueuedEvents:clusterpingengine.cs(286))(0000000000D28422)No events to process
TL_VERBOSE... (OutboundRouting,ClusterPingEngine.ScheduleTimer:clusterpingengine.cs(349))(0000000000D28422)Exit

A supposed PSTN call was not possible because the SIP Domains were diferent ?!
After some more ‘reverse engeneering’ the outboundrouting process (users with the same domain will  process the call to the mediation server) I suspect that  some code validation was not doing the right Thing.

Time to open a MS Support ticket and in less than 24 hours I got the answer:
The failure is due to new design change in 2013 Outbound Routing. Product group will publish a server side hotfix within 2 weeks to fix the following issue:
“Target Domain and caller domain not same, sending request on its way…” In this case, caller’s domain and target domain do not match. In the hotfix we will stop doing the domain check.

Less than 2 weeks, Microsoft released Lync server Cumulative Update (CU6)  but didn’t find an exact reference to my issue except the KB2995173.
Next step: download CU6, apply on the pre-production environment and check:

TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.constructor:outboundroutingtransaction.cs(382))Creating a OutboundRoutingTransaction object (3)
TL_VERBOSE (OutboundRouting,OutboundRouting.OnRequest:outboundrouting.cs(289))[2359519239]Enter
TL_VERBOSE (OutboundRouting,EmergencyCallHelper.IsEmergencyCall:emergencycallhelper.cs(38))IsEmergencyCall = False
TL_NOISE (OutboundRouting,Settings.GetMatchingVacantNumberEntry:settings.cs(323))Checking for Vacant number entries for +90000
TL_VERBOSE (OutboundRouting,NumberRangeListT<>.GetMatchingRange:numberrangelist.cs(235))(00000000026B0DA2)No matching range found.
TL_NOISE (OutboundRouting,Settings.GetMatchingVacantNumberEntry:settings.cs(363))No matching Vacant Number Range found.
TL_NOISE (OutboundRouting,Settings.GetMatchingCPSEntry:settings.cs(291))Checking for CPS entry for +90000
TL_VERBOSE (OutboundRouting,NumberRangeListT<>.GetMatchingRange:numberrangelist.cs(235))(00000000003C60F1)No matching range found.
TL_NOISE (OutboundRouting,Settings.GetMatchingCPSEntry:settings.cs(316))No matching range found
TL_VERBOSE (OutboundRouting,RoutingHelper.AddHeaderIfNotAlreadyPresent:routinghelper.cs(169))[2359519239]Adding ms-conference header with value true
TL_VERBOSE (OutboundRouting,RoutingHelper.AddHeaderIfNotAlreadyPresent:routinghelper.cs(185))[2359519239]Adding MSReferrerPhone header with value <;user=phone>
TL_VERBOSE (OutboundRouting,RoutingHelper.AddHeaderIfNotAlreadyPresent:routinghelper.cs(169))[2359519239]Adding ms-privacy header with value id
TL_VERBOSE (OutboundRouting,RoutingHelper.RemoveHeaderIfPresent:routinghelper.cs(202))[2359519239]ms-vm-escape-timer header not present request.
TL_VERBOSE (OutboundRouting,EmergencyCallHelper.IsEmergencyCall:emergencycallhelper.cs(38))IsEmergencyCall = False
TL_VERBOSE (OutboundRouting,PhoneRouter.GetRoutes:phonerouter.cs(146))(0000000001F30767)target phone number: +90000, PhoneRouteUsage: chz
TL_VERBOSE (OutboundRouting,PhoneRouter.GetRoutes:phonerouter.cs(215))(0000000001F30767)#hits: 1, route names: CHZ 
TL_VERBOSE (OutboundRouting,PhoneRouter.GetRoutes:phonerouter.cs(218))(0000000001F30767)Exit
TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.SetRouteInformation:outboundroutingtransaction.cs(770))[2359519239]Target is not LBR Restricted; If target has an associated site id then target is a PBX and we may need to enforce PSTN toll bypass rules.
TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.SetRouteInformation:outboundroutingtransaction.cs(788))[2359519239]Target does not have a site assigned to it.  No need to enforce PSTN toll bypass rules.
TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.GetRoutes:outboundroutingtransaction.cs(589))[2359519239]Forking disabled since there are no backup gateways or routes.
TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.GetNextHop:outboundroutingtransaction.cs(890))[2359519239]Attempting to find GetNextHop: Length:1; StartAt:0; Tried:1; Name/State:chze33vd-Up
TL_VERBOSE (OutboundRouting,OutboundGateway.ApplyRulesInList:gateway.cs(475))(0000000001220146)No outbound translation rules defined for Target chze33vd
TL_VERBOSE (OutboundRouting,OutboundGateway.GetTranslatedCallerId:gateway.cs(408))(0000000001220146)No outbound Calling number translation rules defined for target chze33vd
TL_VERBOSE OutboundRouting,OutboundGateway.UpdateCallerId:gateway.cs(381))[2359519239]Updated Caller Id (new PAI =<tel:+80001>)
TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.SetState:outboundroutingtransaction.cs(445))[2359519239]Set transaction 3 state to ClientTimerRunning
TL_VERBOSE (OutboundRouting,ORTransactionTimer.AddPendingTransaction:ortransactiontimer.cs(213))(000000000004DFF6)Enter
TL_VERBOSE (OutboundRouting,ORTransactionTimer.AddPendingTransaction:ortransactiontimer.cs(242))(000000000004DFF6)Global timer started.
TL_VERBOSE (OutboundRouting,ORTransactionTimer.AddPendingTransaction:ortransactiontimer.cs(249))(000000000004DFF6)Exit
TL_VERBOSE (OutboundRouting,OutboundRouting.OnRequest:outboundrouting.cs(292))[2359519239]Exit
TL_VERBOSE (OutboundRouting,OutboundRoutingTransaction.SetState:outboundroutingtransaction.cs(445))[2359519239]Set transaction 3 state to ClientTerminated
TL_VERBOSE (OutboundRouting,ORTransactionTimer.RemovePendingTransaction:ortransactiontimer.cs(255))(000000000004DFF6)Enter
TL_VERBOSE (OutboundRouting,ORTransactionTimer.RemoveListEntry:ortransactiontimer.cs(190))(000000000004DFF6)Remove list entry. Id: 3
TL_VERBOSE (OutboundRouting,ORTransactionTimer.RemovePendingTransaction:ortransactiontimer.cs(262))(000000000004DFF6)Global timer stopped.
TL_VERBOSE (OutboundRouting,ORTransactionTimer.RemovePendingTransaction:ortransactiontimer.cs(267))(000000000004DFF6)Exit

The routing engine looks much different now but… voila! it works.
Time to plan an CU6 upgrade to 6 pools, 18 Front End Servers, 8 Edge Servers, … :)

To finish this post an explanation about the suspicious ‘user not found’ message. It turns out that since the outboundrouting dispatch the ‘call’ back to the Lync engine, the next step was to forward the invite to the edge since it was dialog between different domains. The  Edge Servers will simply return back because they check that the destination domain was… internal (smart guys)… didn’t o much deeper to understand who put the message received by the Lync Client.

Comments welcome

Lync glitch fun #1

Time to implement a find+replace tool on the development tools  :)


Can Lync fly?

This was a proof-of-concept that I made quiet a time ago (may 2013). I was hoping to perform it again with more data and images, but never got the right time :(
With all the new features of Lync 2013 (mobile devices), one of my curious questions was: can I make a video call using Lync on a plane?

The Cookbook
– A Lync 2013 deployment (front-end + edge servers);
– one computer (my home PC) and a smartphone (Iphone 4) with Lync client
Airfield with 3G network coverage
– A plane and a pilot
– A co-pilot to handle the lync call – we prefered to leave the pilot with is main task ;)
– Clear sky for great images

The outcome
Mission partially acomplished! There was one expected problem an one unexpected failure. 3G coverage while the plane is on air is very unstable (image freezes during transmition and the call failed after several minutes) and I forgot to give record permissions to the ‘land’ user.

So I have to improvise and manage to save one picture from my desktop.

I want to thanks to this guys that made this test possible:
Flight academy Hangar 5, for the plane (and is aso a great place to learn to fly);
– “Captain” Nuno Colaço for the flight.


I don’t think we will have Lync conferencing as an airplane gadget, but it’s will just be matter of time and legislation to allow the use of wireless devices on board.
But the airlines will be looking for this potential. One of our customers where I deployed Lync was evaluating the possibility to use Lync with the pilots Ipad to discuss fligth plans with the main office control room (this while on land, of course).

Other samples of video call from planes:
– An inflight facetime videocall from a commercial flight (2010);
Skype in-flight video call made on MSNBC TV;
– Skype Call and Google + Hangout with Gogo Inflight Wifi (2011).


Get every new post delivered to your Inbox.

Join 139 other followers