Degraded Performance across Command Platform

Final update

This incident has been resolved.

1. Resolved · Oct 21, 12:11 UTC
   This incident has been resolved.
2. Monitoring · Oct 21, 02:05 UTC
   The recent service degradations impacting Rapid7 products have now been mitigated. Current Status: All services have returned to normal operation. We appreciate your patience during the recovery process.
3. Monitoring · Oct 20, 21:29 UTC
   Rapid7 Service Status Update Rapid7 has made significant progress in recovering from the recent service degradations impacting InsightIDR, InsightAppSec, Data Collection Management, InsightCloudSec and InsightVM. Current Status: Most services have returned to normal operation. However, Event Collection and Processing remains delayed within InsightIDR - This may continue to cause lag in alert generation and investigation creation. We appreciate your patience and understanding as we complete the recovery process.
4. Monitoring · Oct 20, 19:51 UTC
   Rapid7 Service Recovery Update Rapid7 is actively working to recover from multiple service degradations affecting InsightIDR, InsightAppSec, Data Collection Management, and InsightVM. Current Impact: - Event Processing (InsightIDR & InsightVM): Customers may experience delays in event processing, which could result in lag for alert generation and investigation creation within InsightIDR, as well as event processing degradation in InsightVM. - User Interface (InsightVM): Customers may experience slowness when loading the InsightVM User Interface. - Data Collection Management: The Insight Agent status displayed in the Data Collection Management UI may show inaccurate information. - InsightAppSec: Customers may experience delays when initiating scans within InsightAppSec.
5. Identified · Oct 20, 19:30 UTC
   Rapid7 Service Recovery Update Rapid7 is actively working to recover from multiple service degradations affecting InsightIDR, InsightAppSec, Data Collection Management, and InsightVM. Current Impact: - Event Processing (InsightIDR & InsightVM): Customers may experience delays in event processing, which could result in lag for alert generation and investigation creation within InsightIDR, as well as event processing degradation in InsightVM. - User Interface (InsightVM): Customers may experience slowness when loading the InsightVM User Interface. - Data Collection Management: The Insight Agent status displayed in the Data Collection Management UI may show inaccurate information. - InsightAppSec: Customers may experience delays when initiating scans within InsightAppSec.
6. Identified · Oct 20, 16:49 UTC
   We are investigating multiple service degradations impacting InsightIDR, InsightAppSec, Data Collection Management and InsightVM. - Event Processing: Customers may experience delays in event processing, resulting in lag for alert generation and investigation creation within InsightIDR. - Event Processing and User Interface: InsightVM customers may experience slowness loading the User Interface as well as Event Processing degradation. We are also investigating intermittent authentication degradation. - API Availability: Degradation of specific API performance within InsightIDR may affect integrations and automated workflows. - Data Collection Management: The Insight Agent status within the Data Collection Management UI may display incorrect information. - InsightAppSec: Customers may experience a delay in starting scans within InsightAppSec. We are actively working to identify the root cause and restore normal performance as quickly as possible.
7. Monitoring · Oct 20, 14:25 UTC
   We continue to monitor InsightIDR Event Processing as we process a queue backlog. We are experiencing a delay in starting scans within InsightAppSec, which we are investigating. We have recovered to healthy state in InsightVM and InsightCloudSec
8. Monitoring · Oct 20, 13:16 UTC
   We continue to monitor InsightIDR Event Processing as we process a queue backlog. We have recovered to healthy state in InsightVM, InsightAppSec and InsightCloudSec
9. Monitoring · Oct 20, 10:55 UTC
   We are actively monitoring our recovery from the earlier mentioned incident, you can expect degraded performance across data processing and user interface throughout the Command Platform.