Autodesk Forge - 将文件作为块上传到 Node JS 中的 BIM 360 存储时出现 504 网关超时 [英] Autodesk Forge - 504 Gateway Timeout when uploading file as chunks to BIM 360 storage in Node JS
问题描述
当我尝试上传块"中的大文件时，出现 504 网关超时在 Autodesk forge 中使用 PUT 对象可恢复 API 端点.
I'm getting a 504 Gateway Timeout when I try to upload a large file in "chunks" using the PUT object resumable API endpoint in Autodesk forge.
我的控制台日志输出如下，显示了 504 响应，以及我发送的 PUT 请求.
My console log output is below, showing the 504 response, along with the PUT request I send.
我尝试了各种方法，都使用 Node 中 forge-apis SDK 中的函数(例如 ObjectsAPI 中的 uploadChunk 方法)) 并使用request、request-promise-native编写各种函数，检查是否使用await/async、promise vs callbacks可以帮助解决问题.
I've tried various methods, both using the functions in the forge-apis SDK in Node, (eg uploadChunk method from ObjectsAPI) and also written various functions using request, request-promise-native, to check if using await/async, promises vs callbacks can help solve the problem.
在所有情况下，我都得到相同的响应 - 日志中的长时间暂停，然后是 504 网关超时 - 这似乎来自 Forge 端.
In all cases, I get the same response - a long pause in the log and then a 504 Gateway Timeout - which appears to be coming from the Forge end.
对于上下文，我的 Node JS 应用程序位于 Heroku 上，并由在客户端本地运行的 python 脚本触发 - 所以 Heroku 是中间人" - 身份验证从客户端通过 python 传递到节点应用程序，然后传递给 Forge 进行身份验证.流程中的其他操作成功，因此我确信身份验证工作正常.
For context, my Node JS application sits on Heroku, and is triggered from a python script running locally on the client - so Heroku is the 'middleman' - authentication is passed from client-side via python to the node app and then passed on to Forge to authenticate. Other operations in the flow are successful so I'm confident authentication is working correctly.
有人可以从我下面的日志中看到可能导致问题的任何内容吗?其他人在分块上传大文件时遇到过类似的 504 问题吗?(在这种情况下约 130MB)
Can anyone see anything from my logs below that could be causing the problem? Anybody else had similar 504 issues with chunked uploads of large files? (~130MB in this case)
第一个日志显示了我的控制台日志输出 - 我以 1 秒的间隔发送 5mb 块，并将该块作为切片的缓冲区对象(字节)发送以匹配 Content-Range
The first log shows my console log output - I'm sending 5mb chunks at 1-second intervals, and sending the chunk as a Buffer object (bytes) sliced to match the Content-Range
2020-06-22T10:42:33.389835+00:00 app[web.1]: Ready to upload chunk...
2020-06-22T10:42:33.389896+00:00 app[web.1]: simulating waiting for 1000 milliseconds
2020-06-22T10:42:34.391034+00:00 app[web.1]: done waiting
2020-06-22T10:42:34.392893+00:00 app[web.1]: contentRange bytes 129999974-130056191/130056192
2020-06-22T10:42:34.393393+00:00 app[web.1]: requestParams {
2020-06-22T10:42:34.393394+00:00 app[web.1]: headers: {
2020-06-22T10:42:34.393395+00:00 app[web.1]: Authorization: 'Bearer eyJhbGciOiJIxxxxxxxxx',
2020-06-22T10:42:34.393396+00:00 app[web.1]: 'Content-Type': 'application/octet-stream',
2020-06-22T10:42:34.393396+00:00 app[web.1]: 'Content-Range': 'bytes 129999974-130056191/130056192',
2020-06-22T10:42:34.393397+00:00 app[web.1]: 'Content-Length': '130056192',
2020-06-22T10:42:34.393397+00:00 app[web.1]: 'Session-Id': '-75601742'
2020-06-22T10:42:34.393397+00:00 app[web.1]: },
2020-06-22T10:42:34.393398+00:00 app[web.1]: uri: 'https://developer.api.autodesk.com/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable',
2020-06-22T10:42:34.393399+00:00 app[web.1]: url: 'https://developer.api.autodesk.com/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable',
2020-06-22T10:42:34.393399+00:00 app[web.1]: method: 'PUT',
2020-06-22T10:42:34.393400+00:00 app[web.1]: body: <Buffer 6b b1 b4 e4 d7 f5 1e 59 a0 07 0c db 68 7d 9e 98 75 8e 5f fc e8 0c e3 78 a0 9c b9 59 3f e9 0f 28 e5 5a f0 eb 75 7b 68 16 3d 4c c1 ca 20 7a ba 67 f0 69 ... 56167 more bytes>,
2020-06-22T10:42:34.393400+00:00 app[web.1]: resolveWithFullResponse: true
2020-06-22T10:42:34.393400+00:00 app[web.1]: }
2020-06-22T10:42:34.393466+00:00 app[web.1]: Ready to upload chunk...
对于每个块的响应如下:
The response comes back like this for each chunk:
2020-06-22T10:43:34.443453+00:00 app[web.1]: Uploading 504 > GATEWAY_TIMEOUT
2020-06-22T10:43:34.443834+00:00 app[web.1]: err: null
2020-06-22T10:43:34.443969+00:00 app[web.1]: res: {
2020-06-22T10:43:34.443969+00:00 app[web.1]: ----"statusCode": 504,
2020-06-22T10:43:34.443970+00:00 app[web.1]: ----"body": "",
2020-06-22T10:43:34.443970+00:00 app[web.1]: ----"headers": {
2020-06-22T10:43:34.443971+00:00 app[web.1]: --------"content-length": "0",
2020-06-22T10:43:34.443971+00:00 app[web.1]: --------"connection": "Close"
2020-06-22T10:43:34.443971+00:00 app[web.1]: ----},
2020-06-22T10:43:34.443972+00:00 app[web.1]: ----"request": {
2020-06-22T10:43:34.443972+00:00 app[web.1]: --------"uri": {
2020-06-22T10:43:34.443973+00:00 app[web.1]: ------------"protocol": "https:",
2020-06-22T10:43:34.443973+00:00 app[web.1]: ------------"slashes": true,
2020-06-22T10:43:34.443973+00:00 app[web.1]: ------------"auth": null,
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"host": "developer.api.autodesk.com",
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"port": 443,
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"hostname": "developer.api.autodesk.com",
2020-06-22T10:43:34.443974+00:00 app[web.1]: ------------"hash": null,
2020-06-22T10:43:34.443975+00:00 app[web.1]: ------------"search": null,
2020-06-22T10:43:34.443975+00:00 app[web.1]: ------------"query": null,
2020-06-22T10:43:34.443975+00:00 app[web.1]: ------------"pathname": "/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable",
2020-06-22T10:43:34.443976+00:00 app[web.1]: ------------"path": "/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable",
2020-06-22T10:43:34.443976+00:00 app[web.1]: ------------"href": "https://developer.api.autodesk.com/oss/v2/buckets/wip.dm.prod/objects/1fad0fad-601b-41b8-bce4-c88edbb353ec.rvt/resumable"
2020-06-22T10:43:34.443977+00:00 app[web.1]: --------},
2020-06-22T10:43:34.443977+00:00 app[web.1]: --------"method": "PUT",
2020-06-22T10:43:34.443977+00:00 app[web.1]: --------"headers": {
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Authorization": "Bearer eyJhbGciOiJxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Content-Type": "application/octet-stream",
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Content-Range": "bytes 129999974-130056191/130056192",
2020-06-22T10:43:34.443978+00:00 app[web.1]: ------------"Content-Length": "130056192",
2020-06-22T10:43:34.443979+00:00 app[web.1]: ------------"Session-Id": "-75601742"
2020-06-22T10:43:34.443979+00:00 app[web.1]: --------}
2020-06-22T10:43:34.443979+00:00 app[web.1]: ----}
2020-06-22T10:43:34.443979+00:00 app[web.1]: }
推荐答案
感谢我的同事 Luis Felipe巴黎 - 在按照 Petr 的建议查看 forge-server-utils 后，我们设法解决了这个问题.
Thanks to my colleague Luis Felipe Paris - we managed to resolve the problem, after looking at forge-server-utils as per Petr's advice.
通过更改我们的 uploadChunk 函数以使用模块 axios 而不是 request 或 request-promise - 我们发现 504 超时错误消失了.
By changing our uploadChunk function to use the module axios instead of request or request-promise - we found the 504 timeout error disappeared.
我们正在使用 npm 模块 request 和 request-promise(即使这些天是旧的/已弃用)，因为它支持从一个地方到另一个地方的管道流，而这正在处理我们正在执行的所有其他请求，除了以单独的块上传可恢复"文件.
We were using the npm module request and request-promise (even though legacy / deprecated these days) as it supports piping streams from one place to another, and this was working with all other requests we were doing, except uploading a 'resumable' file in separate chunks.
我们必须将发送的文件有效负载从 Buffer 对象转换为 ArrayBuffer，并正确切片以匹配 Content-Range每个块.
We had to convert the file payload we were sending from a Buffer object to an ArrayBuffer, and slice it correctly to match the Content-Range of each chunk.
一些较旧的伪造示例使用 request - 所以我想这可能对其他用户有用 - 如果 request 给您带来问题，请尝试 axios
Some of the older forge examples use request - so I guess this might be useful for other users - if request gives you problems, try axios
这篇关于Autodesk Forge - 将文件作为块上传到 Node JS 中的 BIM 360 存储时出现 504 网关超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
