Unable to set Global Teams Upgrade mode to UpgradeToTeams.

When you are moving from an on-premise Skype for Business organization to Teams Only in M365, the final step on the way would be to switch your organizational setting to Teams Only in Coexistence mode.

This is done after you move all users from Skype for Business on-prem and decommission the Skype for Business server infrastructure and remove all your on-prem servers. So, there is nothing left on-premise whatsoever(make sure that this is the case).

If you are hosting multiple custom domains in your tenant, you have to make sure that public DNS is pointed to M365 and Skype for Business Online/Teams.

The following lists the records you need to change:

ModifySipCNAMEsipdir.online.lync.com
ModifylyncdiscoverCNAMEwebdir.online.lync.com
Modify_sipfederationtls._tcpSRV0 0 5061 sipfed.online.lync.com
Modify_sip._tlsSRV0 0 443 sipdir.online.lync.com

When you try to change the Coexistence mode in the Teams Admin portal, you might experience that the setting will not save. No further info is displayed.

The remote powershell command for setting the Coexistence mode will display the following error message:

“This organization cannot be upgraded to TeamsOnly at the tenant level because there is an on-premise deployment of Skype for Business detected in 1 or more of it sip domains”.

Check the public DNS for the domains in question and make sure that the DNS points to M365. When all domains are pointed to M365, you should be able to set Coexistence mode to Teams Only either in the Teams Admin portal or by running the powershell command in a remote powershell session:  

Grant-CsTeamsUpgradePolicy -PolicyName UpgradeToTeams -Global 

The end result showing in this image:

Coexistence mode set to Teams only for the entire organization

Microsoft Teams and Realtime Traffic, How VPN is affecting user Experience.

In these challenging times with a lot of users WFH, there is one thing that comes up as an issue in many cases.

Why are our users experiencing bad Audio and Video quality in Teams (or Skype for Business)?

Many companies has designed their remote access solutions with VPN dependencies, and this has been a perfect solution in many years to secure access to company data when users are on the move. However, times are changing, and with the increasing use of Microsoft Teams (replacing Skype for Business in many cases) problems arise.

This is Microsoft’s recommendations regarding realtime traffic and VPN connections:

VPN

VPNs are typically not designed or configured to support real-time media. We recommend you provide an alternate path that bypasses the VPN for Teams traffic. This is commonly known as split-tunnel VPN. Split tunneling means that traffic won’t traverse the VPN but will go directly to Teams. This change will have a positive impact on quality, but also provides the secondary benefit of reducing load from the VPN devices and the organization’s network.

To implement a split-tunnel, consult with your VPN vendor for the configuration details.

These recommendations also apply for Skype for Business where in use.

Troubleshooting Exchange Event ID 4002 from MSExchange Avalability.

This blogpost is about a strange incident I had with a fresh Exchange 2016 two node DAG.

The environment is a virtual VmWare environment. The case was a customer case where I was hired to migrate from a working Exchange 2013 environment to a new Exchange 2016 deployment. The customer had a relatively simple setup with a single AD site and nothing more.

I installed the new Exchange servers, and configured the environment accordingly setting up DAG and configuring mailflow etc. Proceeded with the pilot users and did some testing to confirm the environment was Ok. Everything checked out Ok, and the customer moved all users from Exchange 2013 to 2016. The 2013 servers were decommissioned and everything was Ok.

After a couple of months we suddenly experienced free-busy problems. The users with mailbox on one node would not be able to see free-busy from users on the other node.. This started happening out of the blue without no changes being done in the environment. We also started to see Event ID 4002 in the server logs on the server trying to do free-busy lookup.

Process 17932: ProxyWebRequest CrossSite from S-1-5-21-1409082233-1343024091-725345543-35887 to https://dagmember02.domain.com:444/EWS/Exchange.asmx failed. Caller SIDs: NetworkCredentials. The exception returned is Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException: Proxy web request failed. —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
— End of inner exception stack trace —
at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)
at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
— End of inner exception stack trace —
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.Proxy.RestService.EndGetUserPhoto(IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.UserPhotos.UserPhotoApplication.EndProxyWebRequest(ProxyWebRequest proxyWebRequest, QueryList queryList, IService service, IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequest.EndInvoke(IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling()
— End of inner exception stack trace —
. Name of the server where exception originated: dagmember01. LID: 43532. Make sure that the Active Directory site/forest that contain the user’s mailbox has at least one local Exchange 2010 server running the Availability service. Turn up logging for the Availability service and test basic network connectivity.
I immediately started to search for a possible solution to this strange behaviour, but there was no working solution to be found anywhere(read through most of the posts on this event ID on the internet :)) All other services on the Exchange environment were working fine, there were no other error messages in the logs indicating something wrong, just this Event ID 4002 from time to time when people where trying to add someone to a meeting using Scheduling assistant.
After quite a while of research and asking a couple of colleagues, the solution suddenly appeared.

A colleague of mine asked me wether or not the customer used templates to create the VM’s. After checking this with the customer, we could confirm this. He then told me that he had been experiencing some strange similar behaviour on Exchange 2007 some years ago, and asked me to check if the servers had unique SID’s. I did so, and discovered that both the new Exchange 2016 servers had identical SID’s. The tool used was the PsGetSID from Microsoft Sysinternals.
Turned out that the servers were created from a template in VmWare and not sysprepped. After removing one of the servers and reinstalling it, everything started working fine again.
Bottom line is:
If your Exchange servers start acting weird and there doesn’t seem to be a logical explanation to the problem, check the server SID’s. They have to be unique, or obviously, strange things can start to happen in your environment. In my case, there was no obvious reason to the problems that suddenly started to appear and the server setup was made in good faith 🙂
This might bee a noob fault, but I can imagine that someone else but me would have experienced this or other strange problems with no logical explanation, so I think the tip would be useful in case everything else leads nowhere.
The weird part here is the servers functioning 100% ok for a couple of months before the problems started. I’ve never experienced this before, so for all I know that’s how Exchange handles this kind of misconfiguration?

Skype for Business not able to share PPT presentation with federated participants.

When you experience problems sharing PPT content in a Skype for Business meeting, the reason could be a variety of different factors not beeing implemented in the correct way.

There are many ways to publish the Office Web App server necessary to enable this feature. One of those are the IIS ARR web proxy.

In this case the proxy is set up and everything is working fine with all web services, and the officewebapp URL test(https://officewebapp.yourdomain.com/hosting/discovery) is also returning the correct XML page. However, when an internal user uploads a PPT to the meeting, the federated participant experiences just a spinning “Loading” and the presentastion does not load correctly in the client. Other internal users are able to see the presentation in the meeting.

PPT_Loading

Solution:
Remove response header added from IIS on Revers Proxy server, Open Internet Information Services (IIS) Manager on IIS ARR server.

  • In the Connections pane on the left side, expand the Sites folder and select the site(Default Web Site).
  • Double-click the HTTP Response Headers icon in the feature list in the middle.
  • Highlight “X-Frame-Options” In the Actions pane on the right side, click Remove.
  • Click OK to save your changes.
  • Run iisreset from an elevated commandprompt.

After Removing the X-Frame-Options the presentations renders perfectly from the OfficeWebApp server for external participants as well.

Microsoft Teams Direct Routing GA

msunified.net

Today Microsoft Teams Direct Routing was announced as General Available. This is the means for you to bring your own SIP trunk to Microsoft Teams using only a standard SBC. Today AudioCodes and Ribbon are certified SBC’s for Direct Routing and more are in the works. There are three flavors to Direct Routing

Hosted in Azure!

Yes you read correct. AudioCodes has a certified SBC that now is supported in Azure, which means you can run you Direct Routing SBC in Azure as an appliance.

DRGA6.PNG

Installed in your datacenter connected to your PBX or SIP…

View original post 237 more words

Skype for Business, “SIP/2.0 504 Server time-out” when trying to federate.

I recently came across a small chalenge which maybe is nothing to write about, but I choose to anyway as I came across a few solutions to this error message while I was investigating it.

The problem is the following message in the client log when trying to federate in a fresh Skype for Business on-prem environment.

Message:
SIP/2.0 504 Server time-out

ms-diagnostics: 1034;reason=”Previous hop federated peer did not report diagnostic information”;Domain=”partnerDomain.com”;PeerServer=”accessedgeFQDN.partnerDomain.com”;source=”accessedgeFQDN.yourdomain.com”

After some back and forth and checks of firewall rules and port openings, in addition to going over the topology a few times, I stumbled across the solution(which should have been pretty obvious to start with). It turned out that the SRV records for the domain had been registered with typo’s.

Solution:

Make sure you have the correct DNS entries registered in public DNS for your domain zone.
SRV records should be in the format of _sipfederationTLS._tcp.yourdomain.com weight 0 priority 0 port 5061 host accessedge.yourdomain.com.
Make sure you enter just the _sipfederationTLS._tcp part if you do this manually, as the domain name will be appended automatically in the DNS zone.

When you do a Nslookup -q=srv _sipfederationTLS._tcp.yourdomain.com, it should resolve to your access Edge FQDN in public DNS.

Skype4b: Users prompted for password. Security-Kerberos Event ID 4

Had a rather strange experience the other day that I think needs to be written about.

The scenario is as follows:
Clients registered for Skype for Business a long time ago suddenly starts to be prompted for username and password repeatedly. The problem occurs on Skype for Business users logging on to newly deployed Windows workstations/laptops.

The user had no problem logging in before the computer image was refreshed, but afterwards the password prompts started to show up. The request would be for the user to provide username and password in order to contact the certificate service. All certificates checked out to be fine on the client, but it would not receive a certificate from the FrontEnd server/pool.

The environment is a mixed in-place upgrade from Lync 2013 and some new servers on Skype for Business 2015 server.

After a while of troubleshooting without getting anywhere, I came across som strange messages in the event log saying something about Kerberos.

The Event ID 4 occurred in the System log, and the source was Security-Kerberos:

The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server ddsskypefe16$. The target name used was HTTP/”FrontEndPoolFQDN”.domain.com. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (DOMAIN.COM) is different from the client domain (DOMAIN.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.

I started investigating some more around Kerberos authentication in Skype for Business, and found that sometimes when you do an in-place upgrade, the Kerberos authentication breaks and the referenced account is no longer valid(for some reason). So, after hours of troubleshooting without any luck, I proceeded with removing the old Kerberos account and generated a new one using the following PS commands:

New-CsKerberosAccount -UserAccount “Domain\skypeauth” -ContainerDN “CN=Users,DC=domain,DC=com”

New-CsKerberosAccountAssignment -UserAccount “Domain\skypeauth” -Identity “Site:Main SIte”

Enable-CsTopology

Set-CsKerberosAccountPassword -UserAccount “Domain\skypeauth”

I then ran a test of the Kerberos account assignment:

Test-CsKerberosAccountAssignment -Identity “Site:Main Site” -Report “c:\atea\kerberos_report.htm” -Verbose

After doing this, It would seem like all users are able to log in

Headset review: Plantronics v8200 UC

Another headset review coming up. This time it’s the new Plantronics Voyager 8200 UC, a boom less headset for the business market aimed at the productive worker needing a headset to block out unwanted noise in an open office environment.

The Plantronics Voyager 8200 UC is such a headset. It’s delivered in a soft pouch together with a USB Bluetooth dongle(always use it when connecting to your PC/Mac), a USB charging cable and a mini jack cable for those times when Bluetooth cannot be used(in flight mode for example). The headset is available in two colors, black(the one showed in this post) and white. It has a quite nice finish with leather and aluminum, and no microphone boom.

The boom less construction is surprisingly effective, and both audio quality during conversations and the reduction of background noise is quite good. The callee is not able to hear any noise in the background when making a call, even in a quite noisy environment. The ANC has two settings when turned on, that is Medium and High. With my limited hearing I’m not able to separate the two 🙂

There are also functions for open mic, mute etc. all controlled from the buttons on the headset. When playing music, you can control play/pause, skip forward and backward etc. Incoming calls are prioritized, and answering calls are done with the buttons on the headset or by the auto answer function if the headset is lying on your desk. This is one of my favourite features with Plantronics headsets designed for UC and certified for Skype for Business. The proximity sensors allows for automatic features as ANC on/off and automatic pause of movies/music and answer/mute calls when the headset is put on or taken off. The automatic disabling of ANC when the headset is not used also acts as a power saving feature.
There are however something to bear in mind when it comes to proximity sensors and machine connectivity. As mentioned earlier in this post, the USB dongle has to be used when connecting to a PC or Mac. That being said, I’ve experienced differences between Mac OSx and Windows operating systems when it comes to operating the headset. Specially the Mac OSx seems somewhat limited in regards to controlling playback of music from the headset. I’m sure this is a problem related to the operating system API and not the headset as it works fine in a Windows Client.

When it comes to playing music or watching movies using this headset, it delivers very good sound(at least in my opinion). When using the BT dongle that comes with the headset, I’ve not experienced any problems with audio playback. Some people has experienced audio delay when streaming audio over Bluetooth, but this is not my impression. I’ve tested audio streaming both with USB dongle on my MacBook and with my iPhone and experienced no audio delay.

The Plantronics Voyager v8200 UC is in my opinion a very good all round headset which delivers excellent performance both when used as a productivity headset and as an entertainment headset streaming music and watching movies. It’s a very good alternative to the Plantronics Voyager Focus UC, which in my opinion has been the best UC headset on the market up until now 🙂