Lync multiparty IM and voice fails

After the Lync 2013 upgrade I made at my company in December, we have been having problems with multiparty IM and voice calls, as well as with the Lync whiteboard feature and other stuff.

We develop software that integrates with Lync (http://www.blueposition.com/lync and http://www.busylight.com/), so our production Lync installation should really be working properly, but I haven’t had the time to look into the problem in details, and no amount of googling the error messages helped.

Today however, my colleague had to debug a problem to one of our software products, that only occurred when the user was in a voice conference, so the Lync issue had to be solved – otherwise he had no way of reproducing the software problem.

The problem was, that multiparty IM conversations looked like they started correctly – the conversation initiator could open a Lync IM window with the participants, and could type messages, but the messages never got to the participants.

Adding whiteboard to a 2-party IM or starting a voice conference would give an error message.

Using Snooper I found this error:

ms-diagnostics: 3062;reason="Internal Error: getMcu failed";
HRESULT="0x80004005(E_FAIL)";processing-cluster="lyncfrontendserverfqdn";
processing-frontend="lyncfrontendserverfqdn";source-user-primary-pool="lyncfrontendserverfqdn";
target-user-primary-pool="lyncfrontendserverfqdn";source="lyncfrontendserverfqdn"

No luck googling for the error message.

Going through entries in the Windows event log on the frontend server, I finally found something that looked related:

McuFactory could not find the pool associated with one of the conference directories.

Failed to read pool FQDN associated with conference directory 4.
Cause: The pool associated with the conference directory does not exist anymore.
Resolution:
Conference directories without a valid pool associated can be deleted using management tools.

and

McuFactory could not be initialized.

Lync Server MCU Factory could not be initialized.
Exception:
Conversion failed when converting from a character string to uniqueidentifier..
Stack:
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Microsoft.Rtc.Server.McuFactory.DbAccessor.EndReplicateConfDirectories(ExecuteAsyncStateBase state, IAsyncResult ar)
   at Microsoft.Rtc.Server.McuFactory.DbAccessor.ExecuteReplicateConfDirectories(ExecuteAsyncStateBase state)
   at Microsoft.Rtc.Server.McuFactory.DbAccessor.ReplicateConfDirectories(Int32 numRows, PackedTable pt, Boolean startup)
   at Microsoft.Rtc.Server.McuFactory.McuFactory.PopulateConferenceDirectories(List`1 cds, Boolean startup)
   at Microsoft.Rtc.Server.McuFactory.McuFactory..ctor().
Cause: Application error.This is a bug.
Resolution:
Look in previous event logs for more information about this error.

I especially liked the Cause message on the second event log message

Anyway, the first message indicated “conference directory 4’”. Using the Lync management shell cmdlet Get-CsConferenceDirectory I could see that something was wrong with that directory:

Identity             : 4
ServiceId            : BackCompatSite-UserServices-1
TargetServerIfMoving :
Id                   : 4
Fingerprint          :
Element              : <ConferenceDirectory Id="4" xmlns="urn:schema:Microsoft.Rtc.Management.Settings.PstnConf.2008">
                         <ServiceId xmlns="urn:schema:Microsoft.Rtc.Management.Deploy.Topology.2008" SiteId="BackCompat
                       Site" RoleName="UserServices" Instance="1" />
                         <UpaSeparator xmlns="urn:schema:Microsoft.Rtc.Management.BaseTypes.2008"></UpaSeparator>
                       </ConferenceDirectory>

Since we are not using Lync conferencing in production, I felt it safe to delete that conference directory. This is done with the Remove-CsConferenceDirectory cmdlet. But that cmdlet had the same problem as the McuFactory component: The conferencing directory didn’t have a home pool assigned.

Using the -force parameter, I was able to delete the conference directory.

After restarting the Lync Server Frontend service I could verify that the event log messages were no longer logged, instead I saw a very nice “Lync Server MCU Factory started.” message.

And now we can do multi-party IM, voice conferencing, and even use the whiteboard in our Lync conversations.