Lot of serviceexposemap.remove running in processlist


#1

Hello all,

Since a few weeks I have a lot (almost 10k) of serviceexposemap.remove processes in running state.
And thus I have a lot of error messages in the logs:

rancher-server_1 | 2019-01-11 10:17:05,284 ERROR [:] [] [] [] [ecutorService-5] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405368253] on [1969997], canceled by [State [purged] is not valid for process [instance.stop:1478103799] on resource [3028814]]
rancher-server_1 | 2019-01-11 10:17:05,313 ERROR [:] [] [] [] [ecutorService-1] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405369002] on [1969999], canceled by [State [purged] is not valid for process [instance.stop:1478103801] on resource [3002777]]
rancher-server_1 | 2019-01-11 10:17:05,339 ERROR [:] [] [] [] [ecutorService-4] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405374905] on [1970010], canceled by [State [purged] is not valid for process [instance.stop:1478103803] on resource [3002777]]
rancher-server_1 | 2019-01-11 10:17:05,364 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405378736] on [1970016], canceled by [State [purged] is not valid for process [instance.stop:1478103805] on resource [3002777]]
rancher-server_1 | 2019-01-11 10:17:05,389 ERROR [:] [] [] [] [ecutorService-2] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405378768] on [1970014], canceled by [State [purged] is not valid for process [instance.stop:1478103807] on resource [3028814]]
rancher-server_1 | 2019-01-11 10:17:05,417 ERROR [:] [] [] [] [ecutorService-5] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405475044] on [1970037], canceled by [State [purged] is not valid for process [instance.stop:1478103809] on resource [3028814]]
rancher-server_1 | 2019-01-11 10:17:05,446 ERROR [:] [] [] [] [ecutorService-1] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405500512] on [1970055], canceled by [State [purged] is not valid for process [instance.stop:1478103811] on resource [3028814]]
rancher-server_1 | 2019-01-11 10:17:05,471 ERROR [:] [] [] [] [ecutorService-4] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405501509] on [1970057], canceled by [State [purged] is not valid for process [instance.stop:1478103813] on resource [3002777]]
rancher-server_1 | 2019-01-11 10:17:05,499 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405560342] on [1970080], canceled by [State [purged] is not valid for process [instance.stop:1478103815] on resource [3028814]]
rancher-server_1 | 2019-01-11 10:17:05,524 ERROR [:] [] [] [] [ecutorService-2] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405560955] on [1970082], canceled by [State [purged] is not valid for process [instance.stop:1478103817] on resource [3002777]]
rancher-server_1 | 2019-01-11 10:17:05,551 ERROR [:] [] [] [] [ecutorService-5] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [serviceexposemap.remove:405565685] on [1970104], canceled by [State [purged] is not valid for process [instance.stop:1478103819] on resource [3002777]]

It looks like most of this process are related to a single environment:

mysql> select account_id, count() from service_expose_map where state = ‘removing’ group by account_id;
±-----------±---------+
| account_id | count(
) |
±-----------±---------+
| 7 | 1 |
| 227665 | 1 |
| 227895 | 4 |
| 232815 | 1 |
| 233618 | 1 |
| 247689 | 1 |
| 610941 | 9077 |
| 717843 | 1 |
| 799889 | 2 |
| 844190 | 12 |
| 844717 | 6 |
| 855925 | 1 |
±-----------±---------+
12 rows in set (0.01 sec)

Is it safe to remove the elements from this table using an SQL query?