Akvo Status - Incident History

Akvo RSR Django Upgrade

Wed, 08 Dec 2021 09:20:30 +0100

Dec 8, 09:20 CET
Completed - The scheduled maintenance has been completed.

Dec 8, 08:30 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Dec 7, 17:00 CET
Scheduled - We are doing a quick upgrade of Django. It should be very a nice improvement to all that happens behind the scenes for RSR. Thanks for your patience!

Report generation bugs

Thu, 04 Nov 2021 13:36:23 +0100

Nov 4, 13:36 CET
Resolved - This incident has been resolved.

Nov 4, 13:32 CET
Monitoring - We have fixed the issue and are monitoring the system.

Nov 4, 12:41 CET
Identified - We've identified the issue preventing the downloading of reports and are fixing it.

Nov 4, 11:22 CET
Investigating - We have identified a problem with generation of reports on Akvo Flow. We are currently investigating the issue.

SSL encryption problem

Thu, 09 Sep 2021 15:16:12 +0200

Sep 9, 15:16 CEST
Resolved - The SSL certificate issues has been resolved. Thanks for your patience.

Sep 9, 14:28 CEST
Update - We continue to investigate the issue with our SSL certificates. Our apologies for the inconvenience. We expect to have it resolved shortly.

Sep 9, 11:13 CEST
Investigating - Our SSL certificate didn't renew automatically as it should. We're investigating and fixing the problem

Akvo Flow maintenance

Sat, 08 May 2021 17:00:31 +0200

May 8, 17:00 CEST
Completed - The scheduled maintenance has been completed.

May 8, 09:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 29, 11:55 CEST
Scheduled - We will be undergoing scheduled maintenance on Akvo Flow. The maintenance will affect the Akvo Flow online workspace and data synchronisation with the Akvo Flow app. The maintenance will not affect data collection on the Akvo Flow app, Akvo Lumen, or any other Akvo tools and will not take longer than one day.

Update server

Mon, 26 Apr 2021 10:00:44 +0200

Apr 26, 10:00 CEST
Completed - The scheduled maintenance has been completed.

Apr 26, 09:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Apr 20, 09:33 CEST
Scheduled - Regular server maintenance. Some disruption expected.

Upgrade of servers

Wed, 16 Sep 2020 09:30:34 +0200

Sep 16, 09:30 CEST
Completed - The scheduled maintenance has been completed.

Sep 16, 09:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 15, 10:53 CEST
Scheduled - We will upgrading the servers and several services will need to be restarted and will be unavailable for 30 minutes.

Only some Akvo Flow will not be available (API and reports).

Migration of Lumen Database

Thu, 10 Sep 2020 11:09:08 +0200

Sep 10, 11:09 CEST
Completed - The scheduled maintenance has been completed.

Sep 10, 09:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Sep 7, 10:01 CEST
Scheduled - We will be migrating the Akvo Lumen database to more scalable solution.

Akvo Lumen will be unavailable for the duration of the migration.

Outage problems related with database

Wed, 05 Aug 2020 18:38:21 +0200

Aug 5, 18:38 CEST
Resolved - This incident has been resolved.

Aug 5, 13:47 CEST
Monitoring - A fix has been implemented and we are monitoring the results.

Aug 5, 08:24 CEST
Update - We are continuing to work on a fix for this issue.

Aug 4, 15:28 CEST
Identified - The issue has been identified and a fix is being implemented.

Possible disruption in Akvo Lumen for investigating performance issues

Tue, 28 Jul 2020 17:00:33 +0200

Jul 28, 17:00 CEST
Completed - The scheduled maintenance has been completed.

Jul 28, 15:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jul 28, 14:24 CEST
Scheduled - We are investigating a performance issue.

This investigation may trigger the performance issue again, so Akvo Lumen performance or available may be affected.

Degraded performance Akvo Lumen

Fri, 17 Jul 2020 12:40:39 +0200

Jul 17, 12:40 CEST
Resolved - This incident has been resolved.

Jul 16, 16:18 CEST
Update - We are continuing to monitor for any further issues.

Jul 16, 16:18 CEST
Monitoring - The issue has been identified and we have made a fix for it and are monitoring the database performance

Jul 16, 12:05 CEST
Investigating - We have noticed degraded performance in our databases that power Akvo Lumen. These result in elements in Lumen not loading or taking a very long time to load. We are currently investigating the cause

RSR outage

Tue, 09 Jun 2020 17:10:03 +0200

Jun 9, 17:10 CEST
Resolved - RSR was automatically restarted

Jun 9, 17:00 CEST
Investigating - We are currently investigating this issue.

Akvo lumen maps service not working

Wed, 03 Jun 2020 15:27:44 +0200

Jun 3, 15:27 CEST
Resolved - This incident has been resolved.

Jun 3, 13:34 CEST
Identified - We have a SSL related problem in lumen-maps service

Flow backend maintenance

Tue, 02 Jun 2020 19:22:55 +0200

Jun 2, 19:22 CEST
Completed - The scheduled maintenance has been completed.

Jun 2, 18:30 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Jun 2, 17:53 CEST
Scheduled - Flow backend update

Akvo Flow maintenance

Fri, 22 May 2020 17:32:27 +0200

May 22, 17:32 CEST
Completed - Upgrade completed

May 22, 17:15 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

May 22, 17:10 CEST
Scheduled - Upgrade of internal service in charge of report generation, bulk upload, cascade import

Invalid errors when importing Spreasheet

Thu, 30 Apr 2020 09:00:00 +0200

Apr 30, 09:00 CEST
Resolved - Valid data cleaning spreadsheet are reported with invalid headers

We are currently experiencing a server https certificate verification error

Fri, 10 Apr 2020 17:33:45 +0200

Apr 10, 17:33 CEST
Resolved - Issue was fixes.

Apr 10, 17:33 CEST
Update - We are continuing to work on a fix for this issue.

Apr 10, 16:35 CEST
Update - We are continuing to work on a fix for this issue.

Apr 10, 15:54 CEST
Identified - the issue is:
cannot verify *.akvolumen.org’s certificate, issued by ‘CN=Let’s Encrypt Authority X3,O=Let’s Encrypt,C=US’

RSR database upgrade

Thu, 02 Apr 2020 19:00:30 +0200

Apr 2, 19:00 CEST
Completed - The scheduled maintenance has been completed.

Apr 2, 17:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Mar 31, 16:38 CEST
Scheduled - RSR database will be upgrade and RSR will not be available during that time.

Email server malfunction

Fri, 21 Feb 2020 12:00:02 +0100

Feb 21, 12:00 CET
Resolved - This incident has been resolved.

Feb 20, 18:27 CET
Monitoring - A fix has been implemented and we are monitoring the results.

Feb 20, 12:39 CET
Identified - We identified the problem and are still working on a fix.

Feb 19, 13:33 CET
Investigating - We have identified an issue with our email server (not sending out emails) that may affect some users e.g. getting notifications of downloaded reports, and signing up new users.

Akvo RSR Server Error

Tue, 11 Feb 2020 13:06:42 +0100

Feb 11, 13:06 CET
Resolved - This incident has been resolved.

Feb 11, 13:03 CET
Investigating - We are currently experiencing a server error. Resolving it as quickly as possible.

Thanks for your patience.

Akvo RSR software update - 4.3 Yellow-billed Loon

Thu, 05 Dec 2019 07:45:30 +0100

Dec 5, 07:45 CET
Completed - The scheduled maintenance has been completed.

Dec 5, 07:16 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Dec 5, 06:37 CET
Scheduled - A bunch of improvements to the new project editor around validation and error reporting.

More details found here: https://github.com/akvo/akvo-rsr/releases/tag/v4.3

Survey publishing issues identified

Fri, 22 Nov 2019 09:42:34 +0100

Nov 22, 09:42 CET
Resolved - This incident has been resolved.

Nov 22, 09:42 CET
Update - We have resolved the issues on Akvo Flow. All users need to:
* Republish survey forms published between 19 and 21 November
* On the Flow app using these affected forms, Re-download all surveys to ensure the latest published versions are downloaded to the app (make sure all data is synced before hand)

Nov 21, 14:24 CET
Monitoring - Affected users will need to republish all forms that were published between 19th Nov and today in order to resolve the issues.

Nov 20, 17:28 CET
Identified - The issue has been identified and a fix is being implemented.

Nov 20, 17:27 CET
Investigating - We have identified a problem with survey publishing as a result of the latest release. It affects a subset of users that use number questions and have published surveys after 19/11/2019. We are working to rectify this as soon as possible.

Akvo RSR software update - 4.2 Orange-footed Scrubfowl

Thu, 14 Nov 2019 09:30:34 +0100

Nov 14, 09:30 CET
Completed - The scheduled maintenance has been completed.

Nov 14, 09:00 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Nov 14, 08:32 CET
Scheduled - A bunch of improvements to the new project editor, including improved support for custom fields.

More details found here: https://github.com/akvo/akvo-rsr/releases/tag/v4.2

Akvo RSR software update - 4.1 Spoon-billed Sandpiper

Tue, 29 Oct 2019 15:00:30 +0100

Oct 29, 15:00 CET
Completed - The scheduled maintenance has been completed.

Oct 29, 14:15 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Oct 29, 14:00 CET
Scheduled - A bunch of improvements to the new project editor and the my projects listing page.

More details found here: https://github.com/akvo/akvo-rsr/releases/tag/v4.1

Akvo Flow software update

Mon, 28 Oct 2019 17:15:30 +0100

Oct 28, 17:15 CET
Completed - The scheduled maintenance has been completed.

Oct 28, 14:15 CET
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Oct 28, 13:40 CET
Scheduled - Today we will be updating Akvo Flow with the new login page: https://flowsupport.akvo.org/article/show/95589-new-login

Akvo Flow software update

Thu, 17 Oct 2019 18:00:35 +0200

Oct 17, 18:00 CEST
Completed - The scheduled maintenance has been completed.

Oct 17, 13:00 CEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.

Oct 17, 12:35 CEST
Scheduled - Today we will be updating Akvo Flow with the latest version, 1.10. You might experience some downtime on the online workspace. Check our Data platform updates page on what is new: https://akvo.org/data-platform-updates/