[Bug]: files shared with a team do not sync (ETag cache is not invalidated)

[Ornanovitch]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

I am part of a team and as such I have access to a shared directory with this team. For some time now, when files are added or deleted in this directory, clients do not synchronise properly (sometimes they do not synchronise at all).

After investigating, it seems to be a cache issue: when updating files shared with a team, ETag is updated in DB but is not when PROPFIND from a team user.

sudo -u www-data php /var/www/nextcloud/occ memcache:distributed:clear resolves the issue (see #57973 (comment))

Steps to reproduce

1. As user 1, create a team and add user B into it
2. Create a folder and share it with the team
3. As user B, ensure the folder is synced as a sub-folder of the synced root (do not add it as a new sync folder)
4. As user A or B (it does not matter), perform operations in the directory (add files, delete others) from the web interface
5. Verify that the folder's ETag is different between curl and the database

Expected behavior

Everything is synced well on the computer because the new ETag is propagated

Nextcloud Server version

32

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.4

Web server

Other

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 31 to 32)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

Details

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "brume.felinn.org"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "32.0.5.0",
        "overwrite.cli.url": "https:\/\/brume.felinn.org",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "default_language": "fr",
        "default_locale": "fr_FR",
        "default_phone_region": "FR",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED",
            "HTTP_FORWARDED_FOR",
            "HTTP_X_FORWARDED_FOR"
        ],
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "user": "default",
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance_window_start": 1,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpauth": true,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "skeletondirectory": "\/etc\/nextcloud\/skeleton",
        "enabledPreviewProviders": [
            "OC\\Preview\\MP3",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\OpenDocument",
            "OC\\Preview\\Krita",
            "OC\\Preview\\Imaginary",
            "OC\\Preview\\ImaginaryPDF",
            "OC\\Preview\\Movie"
        ],
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "preview_format": "webp",
        "preview_max_x": 1980,
        "preview_max_y": 1980,
        "versions_retention_obligation": "auto, 30",
        "trashbin_retention_obligation": "auto, 30",
        "forbidden_filenames": [
            ".DS_Store"
        ],
        "theme": "",
        "loglevel": 2,
        "log_type": "file",
        "updater.release.channel": "stable",
        "activity_use_cached_mountpoints": true,
        "maintenance": false,
        "app_install_overwrite": [
            "passman"
        ]
    }
}

List of activated Apps

Details

- activity: 5.0.0-dev.0
  - announcementcenter: 7.3.0
  - bookmarks: 16.0.1
  - calendar: 6.1.5
  - circles: 32.0.0
  - cloud_federation_api: 1.16.0
  - collectives: 3.5.0
  - comments: 1.22.0
  - contacts: 8.3.1
  - contactsinteraction: 1.13.1
  - csp_editor: 1.8.0
  - dav: 1.34.2
  - deck: 1.16.3
  - event_update_notification: 2.8.0
  - federatedfilesharing: 1.22.0
  - files: 2.4.0
  - files_downloadlimit: 5.0.0-dev.0
  - files_lock: 32.0.1
  - files_pdfviewer: 5.0.0-dev.0
  - files_reminders: 1.5.0
  - files_sharing: 1.24.1
  - files_trashbin: 1.22.0
  - files_versions: 1.25.0
  - groupfolders: 20.1.9
  - intros: 1.2.1
  - logreader: 5.0.0-dev.0
  - lookup_server_connector: 1.20.0
  - music: 2.5.1
  - news: 27.2.0
  - nextcloud_announcements: 4.0.0-dev.0
  - notes: 4.12.4
  - notifications: 5.0.0-dev.0
  - notify_push: 1.3.0
  - oauth2: 1.20.0
  - passman: 2.4.12
  - photos: 5.0.0-dev.1
  - polls: 8.6.3
  - privacy: 4.0.0-dev.0
  - profile: 1.1.0
  - provisioning_api: 1.22.0
  - related_resources: 3.0.0-dev.0
  - richdocuments: 9.0.2
  - serverinfo: 4.0.0-dev.0
  - settings: 1.15.1
  - sharebymail: 1.22.0
  - tables: 1.0.3
  - tasks: 0.17.1
  - text: 6.0.1
  - theming: 2.7.0
  - theming_customcss: 1.19.0
  - twofactor_backupcodes: 1.21.0
  - updatenotification: 1.22.0
  - user_oidc: 8.3.0
  - viewer: 5.0.0-dev.0
  - webhook_listeners: 1.3.0
  - workflowengine: 2.14.0

Nextcloud Logs

Absolutely no logs on the server! And no error from the client.

Additional info

Other kinds of sharing (personal share, groupfolders) do sync well

[Ornanovitch]

A workaround, if your team is small enough: share the resource with its members individually...

[Ornanovitch]

Today I found a consistent pattern, see the Edit note on the OP: it seems to be a client issue

[Ornanovitch]

Wow, in fact maybe this is a server issue...

Directory ETag before adding a new file:

❯ curl -X PROPFIND -u "{user}:{pass}" \
        -H "Depth: 0" \
        "https://{server}/remote.php/dav/files/{user}/{THE TEAM SHARED FOLDER}/" \
        --data '<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:prop><d:getetag/></d:prop></d:propfind>'
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns"><d:response><d:href>/remote.php/dav/files/{user}/{THE TEAM SHARED FOLDER}/</d:href><d:propstat><d:prop><d:getetag>&quot;697d08526533c&quot;</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>

Same request after adding a new file:

❯ curl -X PROPFIND -u "{user}:{pass}" \
        -H "Depth: 0" \
        "https://{server}/remote.php/dav/files/{user}/{THE TEAM SHARED FOLDER}/" \
        --data '<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:prop><d:getetag/></d:prop></d:propfind>'
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns"><d:response><d:href>/remote.php/dav/files/{user}/{THE TEAM SHARED FOLDER}/</d:href><d:propstat><d:prop><d:getetag>&quot;697d08526533c&quot;</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>

Same ETag : 697d08526533c = no sync, is that right?

Should I transfer this issue back to server? @ArtificialOwl @AndyScherzinger I imagine you could tell

[AndyScherzinger]

I can't but @ArtificialOwl might or @sorbaugh / @artonge for the team

[Ornanovitch]

I confirm this is a server issue, and more specifically some sort of cache issue, because ETag is correctly updated in db, but it is not returned with PROPFIND :

SELECT fileid, path, etag, storage_mtime  FROM oc_filecache  WHERE path LIKE '%{THE TEAM SHARED FOLDER}'  ORDER BY path;

+---------+---------------------------------+---------------+---------------+
| fileid  | path                            | etag          | storage_mtime |
+---------+---------------------------------+---------------+---------------+
| 4065026 | files/{THE TEAM SHARED FOLDER}  | 697dd8a8ceffd |    1769855138 |

❯ curl -X PROPFIND -u "{user}:{pass}" \
        -H "Depth: 0" \
        "https://{server}/remote.php/dav/files/{user}/{THE TEAM SHARED FOLDER}/" \
        --data '<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:prop><d:getetag/></d:prop></d:propfind>'
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns"><d:response><d:href>/remote.php/dav/files/{user}/{THE TEAM SHARED FOLDER}/</d:href><d:propstat><d:prop><d:getetag>&quot;697d16281d7a2&quot;</d:getetag></d:prop><d:status>HTTP/1.1 200 OK</d:status></d:propstat></d:response></d:multistatus>

Clearing the memcache solves the problem immediately:

sudo -u www-data php /var/www/nextcloud/occ memcache:distributed:clear

I transfer this back to server, sorry for the noise...