Skip to content

Conversation

@jcortes
Copy link
Collaborator

@jcortes jcortes commented Oct 29, 2025

… dropped

WHY

Resolves #18884

Summary by CodeRabbit

  • Bug Fixes

    • "New email in shared folder" events now order and filter by sent time, persist sent time for progress tracking, include consistent event timestamps, use the message ID for events, and display email subjects in summaries.
  • Chores

    • Updated Microsoft Outlook component to v1.7.4 and the "New email in shared folder" source to v0.0.4.

@jcortes jcortes self-assigned this Oct 29, 2025
@vercel
Copy link

vercel bot commented Oct 29, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Preview Comments Updated (UTC)
pipedream-docs Ignored Ignored Nov 6, 2025 4:13pm
pipedream-docs-redirect-do-not-edit Ignored Ignored Nov 6, 2025 4:13pm

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 29, 2025

Walkthrough

Bumps package and module versions and updates the shared-folder email source to use sentDateTime for querying/ordering and timestamping, persist lastDate from sentDateTime, emit events with id = item.id, include ts from sentDateTime, and use the email subject in summaries.

Changes

Cohort / File(s) Change Summary
Package version update
components/microsoft_outlook/package.json
Version bumped from 1.7.3 to 1.7.4.
Shared-folder source update
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
Module version bumped 0.0.30.0.4; query/order switched to sentDateTime (order by sentDateTime desc; filter sentDateTime gt lastDate); compute ts = Date.parse(item.sentDateTime) and persist lastDate from first item's sentDateTime; emit uses id = item.id, includes ts in payload, and summaries reference the email subject.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Poller
  participant Source as new-email-in-shared-folder
  participant OutlookAPI as Microsoft Outlook API
  Poller->>Source: trigger poll
  Source->>OutlookAPI: listSharedFolderMessages(filter: "sentDateTime gt lastDate", orderBy: "sentDateTime desc")
  OutlookAPI-->>Source: items[]
  alt items present
    loop each item
      rect rgb(230,247,255)
        Source->>Source: ts = Date.parse(item.sentDateTime)
        Source->>Source: eventId = item.id
      end
      Source-->>Poller: emit { id: eventId, ts, item, subject, ... }
    end
  else no items
    Source-->>Poller: no events
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Verify Date.parse(item.sentDateTime) handles timezones and invalid/missing values.
  • Confirm lastDate persistence and pagination when multiple items share identical sentDateTime.
  • Ensure switching dedup key to item.id resolves thread dedup issue and consider cases with missing/duplicate id.

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Description check ❓ Inconclusive The description is minimal and lacks detail about what was changed and why, though it does reference the linked issue resolving the problem. Expand the description to explain that ConversationID was replaced with item.id as the deduplication key and why (to prevent duplicate filtering of replies in the same thread).
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically identifies the fix: changing the deduplication key from ConversationID to a per-message identifier in the Outlook Shared Email source to prevent event loss.
Linked Issues check ✅ Passed The PR successfully addresses issue #18884 by replacing ConversationID with item.id as the deduplication key and updating query logic to use sentDateTime, fully resolving the reported bug.
Out of Scope Changes check ✅ Passed All changes (version bumps, deduplication key fix, query logic updates, timestamp handling) are directly related to fixing the deduplication issue and are appropriately scoped to the issue requirements.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch fix-ms-outlook-source-with-wrong-dedupe-key

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (1)

72-72: Consider updating the summary message to use item.id.

The summary message still references conversationId, which may be confusing since the deduplication key now uses item.id. For consistency, consider updating the summary to reference the unique message ID:

-            summary: `A new email with id: "${item.conversationId}" was received!`,
+            summary: `A new email with id: "${item.id || item.conversationId}" was received!`,
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6da388e and bb688eb.

📒 Files selected for processing (2)
  • components/microsoft_outlook/package.json (1 hunks)
  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: pnpm publish
  • GitHub Check: Lint Code Base
  • GitHub Check: Publish TypeScript components
  • GitHub Check: Verify TypeScript components
🔇 Additional comments (3)
components/microsoft_outlook/package.json (1)

3-3: LGTM!

The version bump from 1.7.3 to 1.7.4 is appropriate for this bug fix.

components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (2)

9-9: LGTM!

The module version bump from 0.0.3 to 0.0.4 aligns with the package version update.


67-67: LGTM!

Extracting createdDateTime to a local variable improves code clarity and ensures consistent usage.

item,
{
id: item.conversationId,
id: `${item.id || item.conversationId}-${ts}`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify that item.id is always present and consider consistency with other sources.

The fix correctly addresses the deduplication issue by using item.id (unique per message) instead of item.conversationId (shared across thread). However, there are a few concerns:

  1. Fallback risk: The fallback to conversationId when item.id is falsy could reintroduce the original bug for messages without an id. Please verify that the Outlook shared folder messages API always returns item.id.

  2. Timestamp necessity: Appending the timestamp seems unnecessary if item.id is truly unique per message. According to issue [BUG] Dedup key in Outlook Shared Email Received causing events to be dropped #18884, other Outlook sources use item.id alone. Consider whether this timestamp appending is needed or if it introduces inconsistency across Outlook sources.

Run the following script to check how other Outlook sources handle deduplication:

Also, please verify the structure of the Outlook shared folder messages API response to confirm item.id is always present:


🏁 Script executed:

#!/bin/bash
# Check deduplication patterns in other Outlook sources

# Find all Outlook source files
fd -e mjs -e js 'sources' components/microsoft_outlook/

# Search for email event emission patterns in Outlook sources
rg -nP -A5 -B5 'this\.\$emit\s*\(' components/microsoft_outlook/sources/ -g '*.mjs' -g '*.js'

Length of output: 4667


🌐 Web query:

Does the Microsoft Graph API for listing messages in shared folders always return an id field for each message?

💡 Result:

Short answer: No — it should, but not always in practice.

Details:

  • The message resource includes id as a required/read-only property and List messages normally returns an id for each message. [1]
  • However, there have been intermittent/known service bugs where Graph responses for mail (and other Outlook/Graph resources) return empty or missing id (or other fields) for some items (reported on StackOverflow / Microsoft forums and Tech Community). If you see it, it’s a server-side issue and not expected behavior of the API. [2][3]

If you want, tell me an example response/request and I can help diagnose (permission, endpoint used for shared mailbox, or whether Prefer: IdType=ImmutableId would help).

Sources:

  1. Microsoft Graph — message resource (properties include id). [1]
  2. StackOverflow / developer reports of empty id for Outlook resources. [2]
  3. Microsoft Tech Community / Q&A reports of missing fields in Graph responses. [3]


🏁 Script executed:

#!/bin/bash
# Check generateMeta implementations in Outlook sources

# Find generateMeta in new-email.mjs
echo "=== new-email.mjs generateMeta ==="
rg -nA10 'generateMeta' components/microsoft_outlook/sources/new-email/new-email.mjs

# Check if there's a base class
echo -e "\n=== Looking for parent classes/mixins ==="
rg -nP 'class|extends|mixins' components/microsoft_outlook/sources/new-email/new-email.mjs | head -5

# Find any base source class
echo -e "\n=== Checking new-email-in-shared-folder parent ==="
rg -nP 'class|extends|mixins' components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs | head -5

# Look for where generateMeta is defined if not in new-email.mjs
echo -e "\n=== Searching for base generateMeta definition ==="
rg -nA5 'generateMeta\s*\(' components/microsoft_outlook/sources/ -g '*.mjs' | head -30

Length of output: 3390


🌐 Web query:

Microsoft Graph message id field immutable outlook shared folder API

💡 Result:

Short answer: You can get an immutable message id via Microsoft Graph, but you must opt in (Prefer: IdType="ImmutableId") on every request and access the message in the owning mailbox (shared-folder access uses the owner's mailbox endpoints). Immutable IDs remain stable while the item stays in the same mailbox (moving between folders in that mailbox does NOT change the immutable id), but they will change if the item is moved to an archive mailbox or exported/re‑imported. See Microsoft Graph docs for shared/delegated folder access and for the ImmutableId behavior. [1][2]

Sources:

  • Accessing messages in shared/delegated folders (use users/{owner}/mailfolders/... and Mail.Read.Shared / Mail.ReadWrite.Shared permissions). [1]
  • Immutable IDs: Prefer: IdType="ImmutableId", lifetime rules, and translateExchangeIds guidance. [2]

Remove timestamp and fallback; align with new-email.mjs pattern using item.id directly.

The verification reveals a critical inconsistency. The related new-email.mjs source handles deduplication using md5(item.id) without a fallback or timestamp. The current fix at line 71 introduces both a fallback and timestamp, which is:

  1. Inconsistent: No other Outlook email source uses timestamp appending; new-email.mjs uses md5(item.id) directly with no fallback
  2. Defeats the fix: The fallback to conversationId could reintroduce the original bug if item.id is ever falsy
  3. Unnecessary complexity: Microsoft Graph API message resource includes id as a required/read-only property, so the fallback is defensive for a case that shouldn't occur in normal operation

Recommended fix: Replace line 71 with either id: md5(item.id) (matching new-email.mjs logic) or id: item.id depending on your deduplication requirements. Remove the timestamp—it doesn't provide meaningful deduplication if the primary id field is unreliable, and it breaks consistency with other sources.

🤖 Prompt for AI Agents
In
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
around line 71, the id generation currently uses a fallback and timestamp
(`${item.id || item.conversationId}-${ts}`) which is inconsistent with
new-email.mjs and can reintroduce the original bug; remove the timestamp and
fallback and set the id to match the other source (preferably use the same
dedupe strategy: md5(item.id)) so replace the current expression with a direct
md5(item.id) (or item.id if you choose not to hash) and ensure any required md5
import/usage aligns with new-email.mjs.

GTFalcao
GTFalcao previously approved these changes Oct 29, 2025
@vunguyenhung
Copy link
Collaborator

@vunguyenhung
Copy link
Collaborator

Hello everyone, I have tested this PR and there're some test cases failed or needed improvement.

Please check test reports below for more information:

@jcortes jcortes force-pushed the fix-ms-outlook-source-with-wrong-dedupe-key branch from bb688eb to eecb53d Compare November 4, 2025 19:39
@jcortes
Copy link
Collaborator Author

jcortes commented Nov 4, 2025

Hi @vunguyenhung I've just pushed the fix, please try again! thanks!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (1)

68-74: Critical: Still using conversationId instead of item.id for deduplication.

The current implementation uses ${item.conversationId}-${ts} at line 72, which still relies on conversationId (the thread-level identifier) as the base. This doesn't fully address issue #18884, which explicitly states that item.id (the per-message identifier) should be used instead, as done in other Outlook sources.

While appending the timestamp makes messages within the same thread unique, this approach:

  1. Remains inconsistent with other Outlook email sources like new-email.mjs, which use md5(item.id) directly
  2. Doesn't follow the recommended pattern from issue #18884
  3. May contribute to the test failures mentioned by vunguyenhung

Apply this diff to align with the pattern used in new-email.mjs and fully resolve the issue:

-      for (const item of responseArray.reverse()) {
-        const ts = Date.parse(item.createdDateTime);
-        this.$emit(
-          item,
-          {
-            id: `${item.conversationId}-${ts}`,
-            summary: `A new email with id: "${item.conversationId}" was received!`,
-            ts,
-          },
-        );
-      }
+      for (const item of responseArray.reverse()) {
+        const ts = Date.parse(item.createdDateTime);
+        this.$emit(
+          item,
+          {
+            id: item.id,
+            summary: `A new email with id: "${item.id}" was received!`,
+            ts,
+          },
+        );
+      }

Note: If you want to hash the id for consistency with new-email.mjs, import and use md5 from md5 package and set id: md5(item.id).

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bb688eb and eecb53d.

📒 Files selected for processing (2)
  • components/microsoft_outlook/package.json (1 hunks)
  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • components/microsoft_outlook/package.json
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-07-09T18:07:12.426Z
Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 17538
File: components/aircall/sources/new-sms/new-sms.mjs:19-25
Timestamp: 2025-07-09T18:07:12.426Z
Learning: In Aircall API webhook payloads, the `created_at` field is returned as an ISO 8601 string format (e.g., "2020-02-18T20:52:22.000Z"), not as milliseconds since epoch. For Pipedream components, this needs to be converted to milliseconds using `Date.parse()` before assigning to the `ts` field in `generateMeta()`.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: pnpm publish
  • GitHub Check: Publish TypeScript components
  • GitHub Check: Verify TypeScript components
  • GitHub Check: Lint Code Base
🔇 Additional comments (1)
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (1)

9-9: LGTM: Version bump is appropriate.

The version increment follows semantic versioning for a bug fix.

@vunguyenhung
Copy link
Collaborator

Hello everyone, I have tested this PR and there're some test cases failed or needed improvement.

Please check test reports below for more information:

@jcortes jcortes force-pushed the fix-ms-outlook-source-with-wrong-dedupe-key branch from eecb53d to b6f6b8e Compare November 5, 2025 17:07
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (1)

73-73: Critical: Deduplication key still uses conversationId instead of item.id.

The current implementation uses conversationId + timestamp, which only partially addresses issue #18884. This approach has significant problems:

  1. Root cause not fixed: conversationId still identifies the thread, not individual messages
  2. Timestamp collision risk: Multiple emails in the same thread arriving at the exact same second could still be deduplicated incorrectly
  3. Inconsistent with other sources: According to the past review comment, new-email.mjs uses md5(item.id) for deduplication without fallback or timestamp
  4. Defeats the PR objective: The issue clearly states the fix should use item.id (per-message identifier) instead of conversationId (per-thread identifier)

Apply this diff to properly fix the deduplication issue:

-            id: `${item.conversationId}-${ts}`,
+            id: item.id,

Or to match the pattern in new-email.mjs, import md5 at the top and use:

+import md5 from "md5";
+
 ...
-            id: `${item.conversationId}-${ts}`,
+            id: md5(item.id),

Note: This was already flagged in the previous review. The Microsoft Graph API message resource includes id as a required/read-only property, so item.id should always be present. Based on learnings

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between eecb53d and b6f6b8e.

📒 Files selected for processing (2)
  • components/microsoft_outlook/package.json (1 hunks)
  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (3 hunks)
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 14265
File: components/the_magic_drip/sources/common.mjs:35-43
Timestamp: 2024-10-10T19:18:27.998Z
Learning: In `components/the_magic_drip/sources/common.mjs`, when processing items in `getAndProcessData`, `savedIds` is intentionally updated with IDs of both emitted and non-emitted items to avoid emitting retroactive events upon first deployment and ensure only new events are emitted as they occur.
📚 Learning: 2025-01-29T22:59:38.825Z
Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 15436
File: components/printful/printful.app.mjs:55-63
Timestamp: 2025-01-29T22:59:38.825Z
Learning: Console.log statements should be removed before merging PRs to maintain code quality and prevent potential security risks from exposing sensitive information in logs.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
📚 Learning: 2025-02-05T21:58:03.118Z
Learnt from: jverce
Repo: PipedreamHQ/pipedream PR: 15479
File: packages/connect-react/src/components/ComponentForm.tsx:23-24
Timestamp: 2025-02-05T21:58:03.118Z
Learning: In the connect-react package, the `enableDebugging` property should be of type `boolean` as it's used for toggling debugging features and conditional rendering.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: pnpm publish
  • GitHub Check: Publish TypeScript components
  • GitHub Check: Verify TypeScript components
  • GitHub Check: Lint Code Base
🔇 Additional comments (3)
components/microsoft_outlook/package.json (1)

3-3: LGTM - Appropriate version bump for bug fix.

The patch version increment follows semantic versioning conventions for a bug fix.

components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (2)

9-9: LGTM - Module version aligned with package bump.

The module version increment is consistent with the package.json version bump.


74-75: Improvement: Summary now includes email subject for better visibility.

Including the email subject in the summary provides better context for users monitoring events. Adding ts to the emitted event is also useful for downstream consumers.

However, note that the ts value depends on sentDateTime, which should be verified per the earlier comment.

id: item.conversationId,
summary: `A new email with id: "${item.conversationId}" was received!`,
ts: item.createdDateTime,
id: `${item.conversationId}-${ts}`,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why rely on conversation ID + timestamp instead of message ID?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @alexpareto I've just changed it to item.id

@jcortes
Copy link
Collaborator Author

jcortes commented Nov 5, 2025

Hi @vunguyenhung it seems like you are refering to the conversation/thread in an email message which I don't think would be the right approach to that source component since the main message might have a lot of nested messages. So because the design of the endpoint list only the ones in the shared folder it would not be a good idea to make requests on the each nested message. I hope it makes sense!

@jcortes jcortes force-pushed the fix-ms-outlook-source-with-wrong-dedupe-key branch 2 times, most recently from 2f970da to fdcdd7c Compare November 5, 2025 19:32
@alexpareto
Copy link

Hi @vunguyenhung it seems like you are refering to the conversation/thread in an email message which I don't think would be the right approach to that source component since the main message might have a lot of nested messages. So because the design of the endpoint list only the ones in the shared folder it would not be a good idea to make requests on the each nested message. I hope it makes sense!

I do think that the event should output a new event when replies are received to the original thread.

For example:
john@example.com
bill@sharedinbox.com

  1. Email 1: john@example.com emails bill@sharedinbox.com (triggers event)
  2. Email 2: bill@shareindbox.com replies back to john@example.com (no event, same conversation id)
  3. Email 3: john@example.com replies back to bill@sharedinbox.com (triggers event, same conversation id, different item id)

That said, with the different item.id I think this should work.

@vunguyenhung
Copy link
Collaborator

Hi everyone, all test cases are passed! Ready for release!

Test reports

@jcortes jcortes force-pushed the fix-ms-outlook-source-with-wrong-dedupe-key branch from fdcdd7c to b78a8f8 Compare November 6, 2025 14:04
@jcortes
Copy link
Collaborator Author

jcortes commented Nov 6, 2025

/approve

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (2)

69-69: Note: Timestamp uses sentDateTime.

The timestamp computation is correct, but this is another location using sentDateTime that should be changed to receivedDateTime (as flagged in the Critical issue at lines 50-51).


50-51: Critical: Still using sentDateTime instead of receivedDateTime for polling.

This issue was flagged in the previous review as Critical. Using sentDateTime for polling creates a critical bug: emails sent long ago but received recently will be missed if sentDateTime < lastDate. Per Microsoft Graph API documentation, receivedDateTime should be used for polling "new inbound mail received by the service."

Change both lines to use receivedDateTime:

  • Line 50: $orderBy: "receivedDateTime desc"
  • Line 51: $filter: 'receivedDateTime gt ${lastDate}'

Also update lines 65 and 69 to use receivedDateTime for consistency.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fdcdd7c and b78a8f8.

📒 Files selected for processing (2)
  • components/microsoft_outlook/package.json (1 hunks)
  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (3 hunks)
🧰 Additional context used
🧠 Learnings (3)
📓 Common learnings
Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 14265
File: components/the_magic_drip/sources/common.mjs:35-43
Timestamp: 2024-10-10T19:18:27.998Z
Learning: In `components/the_magic_drip/sources/common.mjs`, when processing items in `getAndProcessData`, `savedIds` is intentionally updated with IDs of both emitted and non-emitted items to avoid emitting retroactive events upon first deployment and ensure only new events are emitted as they occur.
📚 Learning: 2025-01-29T22:59:38.825Z
Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 15436
File: components/printful/printful.app.mjs:55-63
Timestamp: 2025-01-29T22:59:38.825Z
Learning: Console.log statements should be removed before merging PRs to maintain code quality and prevent potential security risks from exposing sensitive information in logs.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
📚 Learning: 2025-02-05T21:58:03.118Z
Learnt from: jverce
Repo: PipedreamHQ/pipedream PR: 15479
File: packages/connect-react/src/components/ComponentForm.tsx:23-24
Timestamp: 2025-02-05T21:58:03.118Z
Learning: In the connect-react package, the `enableDebugging` property should be of type `boolean` as it's used for toggling debugging features and conditional rendering.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
🔇 Additional comments (5)
components/microsoft_outlook/package.json (1)

3-3: LGTM: Version bump is appropriate.

The patch version increment from 1.7.3 to 1.7.4 correctly follows semantic versioning for this bug fix.

components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (4)

9-9: LGTM: Source version bump is appropriate.

The version increment from 0.0.3 to 0.0.4 is consistent with the package-level version bump and appropriate for this bug fix.


64-66: Good: Added defensive check before updating state.

The conditional check before calling _setLastDate prevents updating state when no items are returned, which is good defensive programming.

Note: The use of sentDateTime on line 65 is part of the Critical issue flagged at lines 50-51.


74-74: LGTM: Summary text improvement.

Including the email subject in the summary makes events more user-friendly and easier to identify.


75-75: LGTM: Including timestamp in emission metadata.

Adding the ts field to the emission metadata is good practice and allows proper event timestamping in the Pipedream platform.

Note: The timestamp value is derived from sentDateTime (line 69), which is part of the Critical issue flagged earlier.

@jcortes jcortes force-pushed the fix-ms-outlook-source-with-wrong-dedupe-key branch from b78a8f8 to f1f3bae Compare November 6, 2025 16:13
@jcortes
Copy link
Collaborator Author

jcortes commented Nov 6, 2025

/approve

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (2)

50-51: Critical issue from previous review remains unresolved: Use receivedDateTime instead of sentDateTime.

The previous review identified that using sentDateTime for polling creates a critical bug where emails sent long ago but received recently will be missed. Per Microsoft Graph API documentation, receivedDateTime (when Exchange received the message) should be used for polling scenarios, not sentDateTime (when the sender sent the message).

This affects ordering (line 50), filtering (line 51), and state persistence (line 65). The bug occurs because sentDateTime can be earlier than receivedDateTime due to network delays, offline clients, or delayed delivery.

Apply this diff to fix the polling logic:

           params: {
-            $orderBy: "sentDateTime desc",
-            $filter: `sentDateTime gt ${lastDate}`,
+            $orderBy: "receivedDateTime desc",
+            $filter: `receivedDateTime gt ${lastDate}`,
           },
       if (responseArray.length) {
-        this._setLastDate(responseArray[0].sentDateTime);
+        this._setLastDate(responseArray[0].receivedDateTime);
       }

Also applies to: 64-66


69-69: Timestamp should use receivedDateTime to align with polling logic.

For consistency with the polling logic (which should use receivedDateTime per the previous comment), the event timestamp should also be computed from receivedDateTime rather than sentDateTime.

Apply this diff:

-      const ts = Date.parse(item.sentDateTime);
+      const ts = Date.parse(item.receivedDateTime);
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b78a8f8 and f1f3bae.

📒 Files selected for processing (2)
  • components/microsoft_outlook/package.json (1 hunks)
  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (3 hunks)
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-01-29T22:59:38.825Z
Learnt from: GTFalcao
Repo: PipedreamHQ/pipedream PR: 15436
File: components/printful/printful.app.mjs:55-63
Timestamp: 2025-01-29T22:59:38.825Z
Learning: Console.log statements should be removed before merging PRs to maintain code quality and prevent potential security risks from exposing sensitive information in logs.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
📚 Learning: 2025-02-05T21:58:03.118Z
Learnt from: jverce
Repo: PipedreamHQ/pipedream PR: 15479
File: packages/connect-react/src/components/ComponentForm.tsx:23-24
Timestamp: 2025-02-05T21:58:03.118Z
Learning: In the connect-react package, the `enableDebugging` property should be of type `boolean` as it's used for toggling debugging features and conditional rendering.

Applied to files:

  • components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Lint Code Base
  • GitHub Check: Publish TypeScript components
  • GitHub Check: Verify TypeScript components
  • GitHub Check: pnpm publish
🔇 Additional comments (3)
components/microsoft_outlook/package.json (1)

3-3: LGTM: Version bump is appropriate.

The patch version increment from 1.7.3 to 1.7.4 correctly follows semantic versioning for this bug fix.

components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs (2)

9-9: LGTM: Version bump is appropriate.

The version increment from 0.0.3 to 0.0.4 correctly reflects the bug fix changes in this source.


74-74: LGTM: Improved summary text.

Using item.subject in the summary provides more meaningful information to users compared to the previous id-based summary.

id: item.conversationId,
summary: `A new email with id: "${item.conversationId}" was received!`,
ts: item.createdDateTime,
id: item.id,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Core deduplication fix is correct.

Using item.id directly (instead of item.conversationId) correctly fixes the bug where replies in the same email thread were being dropped. Each message now has a unique deduplication key, resolving issue #18884.

Optional: Consider md5(item.id) for consistency. The related new-email.mjs source uses md5(item.id) for deduplication. While both approaches work, using md5 here would maintain consistency across Outlook sources.

🤖 Prompt for AI Agents
In
components/microsoft_outlook/sources/new-email-in-shared-folder/new-email-in-shared-folder.mjs
around line 73, the deduplication key currently uses item.id which fixes the bug
but is inconsistent with other Outlook sources that use md5(item.id); update the
dedupe value to use md5(item.id) instead of raw item.id to match new-email.mjs:
import or reference the existing md5 utility, compute md5(item.id) where the id
is assigned, and ensure the rest of the logic uses that hashed string for
deduplication.

@jcortes jcortes merged commit 31fd0e0 into master Nov 6, 2025
10 checks passed
@jcortes jcortes deleted the fix-ms-outlook-source-with-wrong-dedupe-key branch November 6, 2025 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Dedup key in Outlook Shared Email Received causing events to be dropped

5 participants