Upgrade fails during migration due to MongoSocketReadTimeoutException timeout error
search cancel

Upgrade fails during migration due to MongoSocketReadTimeoutException timeout error

book

Article ID: 345716

calendar_today

Updated On:

Products

VMware Telco Cloud Automation

Issue/Introduction

  • Upgrade from TCA 2.2.0 to 2.3.0 fails
  • com.mongodb.MongoSocketReadTimeoutException timeout error seen in /common/logs/upgrade/upgrade.log:

com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message at com.mongodb.internal.connection.InternalStreamConnection.translateReadException(InternalStreamConnection.java:563) at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:448) at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:299) at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:259) at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99) at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:450) at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72) at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:226) at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269) at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131) at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:123) at com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:343) at com.mongodb.operation.CommandOperationHelper.executeCommand(CommandOperationHelper.java:334) at com.mongodb.operation.CommandOperationHelper.executeCommandWithConnection(CommandOperationHelper.java:220) at com.mongodb.operation.FindOperation$1.call(FindOperation.java:731) at com.mongodb.operation.FindOperation$1.call(FindOperation.java:725) at com.mongodb.operation.OperationHelper.withReadConnectionSource(OperationHelper.java:463) at com.mongodb.operation.FindOperation.execute(FindOperation.java:725) at com.mongodb.operation.FindOperation.execute(FindOperation.java:89) at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:196) at com.mongodb.client.internal.MongoIterableImpl.execute(MongoIterableImpl.java:143) at com.mongodb.client.internal.MongoIterableImpl.iterator(MongoIterableImpl.java:92) at com.mongodb.client.gridfs.GridFSFindIterableImpl.iterator(GridFSFindIterableImpl.java:89) at com.vmware.vchs.hybridity.migration.MigrationHelper.migrateObjectstoreFilesFromMongoDBToPostgres(MigrationHelper.java:273) at com.vmware.vchs.hybridity.migration.MigrationHelper.migrateDataForGivenMongoCollectionAndValidate(MigrationHelper.java:122) at com.vmware.vchs.hybridity.migration.MigrationManager.executeMigrationAndValidation(MigrationManager.java:129) at com.vmware.vchs.hybridity.migration.MigrationManager.doMigration(MigrationManager.java:84) at com.vmware.vchs.hybridity.migration.MigrationManager.main(MigrationManager.java:50) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:109) at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:580) at com.mongodb.internal.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:445) ... 26 common frames omitted



Environment

2.2, 2.3

Cause

Upgrade will fail if the MongoDB to PostgreSQL migration application is unable to read from MongoDB within 2 minutes. This can occur in very large environments where the MongoDB may be under heavy load.

Resolution

Workaround:

Option 1:

  • Retrying the Upgrade often succeeds if ran again.

Option 2: 
Update the MONGODB_SOCKET_TIMEOUT using the following steps:

  • Customer access shell on the system where migration failed due to above symptom as a root user.
  • Manually download the upgrade bundle tar ball.
  • Extract the upgrade bundle    
tar -xzf VMware-Telco-Cloud-Automation-upgrade-bundle-2.3.0-21500099.tar.gz
  • Open doMigration.sh using an editor and search for MONGODB_SOCKET_TIMEOUT=120.
  • Currently MongoDB socket timeout for migration app is set as 120 seconds default.
MONGODB_SOCKET_TIMEOUT=120
  • Increase the socket timeout parameter MONGODB_SOCKET_TIMEOUT from 120 to 240 .
MONGODB_SOCKET_TIMEOUT=240

   g. Retry upgrade through shell using the following command:

./vsm-upgrade.sh image/VMware-Telco-Cloud-Automation-image-2.3.0-21500099.img.dist



Powered by Wolken Software