3258 - Fix OutOfMemoryError in the workflow editor when iterating with loop over large number of items

[ivicac]

No description provided.

[Copilot]

Pull Request Overview

This PR fixes an OutOfMemoryError that occurs in the workflow editor when iterating with loops over large numbers of items. The fix involves renaming the message-broker-sync module to message-broker-memory and introducing an asynchronous message broker implementation. Additionally, a job cleanup mechanism is implemented to prevent resource accumulation, and a counter-based mechanism is added to limit task executions in the workflow builder.

Key changes:

• Introduced AsyncMessageBroker and refactored message broker architecture with MemoryMessageBroker interface and AbstractMessageBroker base class
• Added job completion tracking with CountDownLatch and automatic cleanup in JobSyncExecutor
• Implemented max task execution limits to prevent memory exhaustion
• Changed connection context from ThreadLocal to ConcurrentHashMap for better thread safety

Reviewed Changes

Copilot reviewed 32 out of 33 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
settings.gradle.kts	Renamed module from message-broker-sync to message-broker-memory
JobSyncExecutor.java	Added job completion waiting, task execution limiting, and async message handling
JobTestExecutor.java	Added job cleanup in finally block to prevent resource leaks
TestExecutorConfiguration.java	Updated to use AsyncMessageBroker with timeout configurations
WebhookConfiguration.java	Updated to use AsyncMessageBroker instead of SyncMessageBroker
TaskDispatcherJobTestExecutor.java	Updated to use new SyncMessageBroker from memory package
ConnectionContext.java	Changed from ThreadLocal to ConcurrentHashMap for thread safety
AbstractMessageBroker.java	New base class for message broker implementations
AsyncMessageBroker.java	New async implementation using executors
SyncMessageBroker.java	Refactored to extend AbstractMessageBroker
MemoryMessageBroker.java	New interface defining receive method
MessageBrokerFactory.java	New empty interface (appears unused)
CollectionUtils.java	Added snapshot creation to avoid ConcurrentModificationException
InMemoryJobRepository.java	Implemented deleteById method for job cleanup
Various build files	Updated dependencies from message-broker-sync to message-broker-memory

Comments suppressed due to low confidence (1)

server/libs/core/message/message-broker/message-broker-memory/src/main/java/com/bytechef/message/broker/memory/config/MemoryMessageBrokerConfiguration.java:50

• Commented-out code should be removed. If the SyncMessageBroker bean configuration is not needed, delete these lines rather than leaving them commented. If it might be needed in the future, document why in a TODO comment or track it in an issue.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sonarqubecloud]

Quality Gate passed for 'server'

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud