使用 AppEngine GoogleCloud 部署 TensorFlow App 时出错
2019-12-02
765
我已经正确部署了 python3 App building-an-app:
但是现在我尝试部署使用神经网络的应用程序,并且它在本地使用 virtualenv 和 requeriments.txt 文件正常工作:
Flask==1.1.1
numpy==1.17.4
scipy==1.3.2
gensim==3.8.1
tensorflow==2.0.0
但是当我使用“gcloud app deploy”时,5 分钟后出现此消息“正在更新服务 [默认]...”,我收到了此错误:
ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build e281a017-86ee-450f-8291-bac8a7a0df30 status: FAILURE.
Build error details: {"error":{"errorType":"BuildError","canonicalCode":"INVALID_ARGUMENT","errorId":"3883CE8A","errorMessage":"02 Dec 2019 22:12:03 INFO Arguments: ['--parser_script=/usr/local/bin/ftl.par', '--src=', '--entrypoint=', '--name=eu.gcr.io/lullaai-chatbot/app-engine-tmp/app/ttl-2h:8f040a55-3198-4998-beb0-75817f57192c', '--directory=/workspace', '--destination=/srv', '--cache-repository=eu.gcr.io/lullaai-chatbot/app-engine-tmp/build-cache/ttl-7d', '--cache', '--builder-output-path=\"\"', '--additional-directory=/.googleconfig', '--python-cmd=/opt/python3.7/bin/python3.7', '--pip-cmd=/env/bin/python3.7 -m pip', '--venv-cmd=/opt/python3.7/bin/python3.7 -m venv /env', '-v=DEBUG', '--base=eu.gcr.io/gae-runtimes/python37:python37_20191019_3_7_4_RC00']\n02 Dec 2019 22:12:03 INFO Unparsed arguments: ['--name=eu.gcr.io/lullaai-chatbot/app-engine-tmp/app/ttl-2h:8f040a55-3198-4998-beb0-75817f57192c', '--directory=/workspace', '--destination=/srv', '--cache-repository=eu.gcr.io/lullaai-chatbot/app-engine-tmp/build-cache/ttl-7d', '--cache', '--builder-output-path=\"\"', '--additional-directory=/.googleconfig', '--python-cmd=/opt/python3.7/bin/python3.7', '--pip-cmd=/env/bin/python3.7 -m pip', '--venv-cmd=/opt/python3.7/bin/python3.7 -m venv /env', '-v=DEBUG', '--base=eu.gcr.io/gae-runtimes/python37:python37_20191019_3_7_4_RC00']\n02 Dec 2019 22:12:03 INFO Executing ['/usr/local/bin/ftl.par', '--name=eu.gcr.io/lullaai-chatbot/app-engine-tmp/app/ttl-2h:8f040a55-3198-4998-beb0-75817f57192c', '--directory=/workspace', '--destination=/srv', '--cache-repository=eu.gcr.io/lullaai-chatbot/app-engine-tmp/build-cache/ttl-7d', '--cache', '--builder-output-path=\"\"', '--additional-directory=/.googleconfig', '--python-cmd=/opt/python3.7/bin/python3.7', '--pip-cmd=/env/bin/python3.7 -m pip', '--venv-cmd=/opt/python3.7/bin/python3.7 -m venv /env', '-v=DEBUG', '--base=eu.gcr.io/gae-runtimes/python37:python37_20191019_3_7_4_RC00']\nINFO FTL version python-v0.17.0\nINFO Beginning FTL build for python\nINFO FTL arg passed: virtualenv_dir /env\nINFO FTL arg passed: ttl 168\nINFO FTL arg passed: python_cmd /opt/python3.7/bin/python3.7\nINFO FTL arg passed: cache True\nINFO FTL arg passed: virtualenv_cmd virtualenv\nINFO FTL arg passed: entrypoint None\nINFO FTL arg passed: exposed_ports None\nINFO FTL arg passed: pip_cmd /env/bin/python3.7 -m pip\nINFO FTL arg passed: tar_base_image_path None\nINFO FTL arg passed: export_cache_stats False\nINFO FTL arg passed: builder_output_path \"\"\nINFO FTL arg passed: destination_path /srv\nINFO FTL arg passed: sh_c_prefix False\nINFO FTL arg passed: base eu.gcr.io/gae-runtimes/python37:python37_20191019_3_7_4_RC00\nINFO FTL arg passed: cache_key_version v0.17.0\nINFO FTL arg passed: cache_salt \nINFO FTL arg passed: cache_repository eu.gcr.io/lullaai-chatbot/app-engine-tmp/build-cache/ttl-7d\nINFO FTL arg passed: venv_cmd /opt/python3.7/bin/python3.7 -m venv /env\nINFO FTL arg passed: name eu.gcr.io/lullaai-chatbot/app-engine-tmp/app/ttl-2h:8f040a55-3198-4998-beb0-75817f57192c\nINFO FTL arg passed: global_cache False\nINFO FTL arg passed: upload True\nINFO FTL arg passed: fail_on_error True\nINFO FTL arg passed: output_path None\nINFO FTL arg passed: directory /workspace\nINFO FTL arg passed: additional_directory /.googleconfig\nINFO FTL arg passed: verbosity DEBUG\nINFO starting: full build\nINFO starting: builder initialization\nINFO Loading Docker credentials for repository 'eu.gcr.io/gae-runtimes/python37:python37_20191019_3_7_4_RC00'\nINFO Loading Docker credentials for repository 'eu.gcr.io/lullaai-chatbot/app-engine-tmp/app/ttl-2h:8f040a55-3198-4998-beb0-75817f57192c'\nINFO builder initialization took 0 seconds\nINFO starting: build process for FTL image\nINFO starting: checking_cached_interpreter_layer\nINFO starting: check python version\nINFO python version full cmd:\n/opt/python3.7/bin/python3.7 --version\nINFO python version stderr:\n\nINFO
有人知道可能是什么问题吗?
检查构建日志中的错误,我发现有一行写着:
步骤 #1 - “builder”: 因错误 [Errno 12] 退出无法分配内存
所以我将 app.yaml 更改为:
runtime: python37
handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
static_dir: static
# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
script: auto
resources:
cpu: 2
memory_gb: 8
disk_size_gb: 10
但我仍然收到相同的错误 :(
这是构建日志:
ERROR: build step 1 "gcr.io/gae-runtimes/python37_app_builder:python37_20191019_3_7_4_RC00" failed: exit status 1
ERROR
Finished Step #1 - "builder"
Step #1 - "builder": IOError: [Errno 2] No such file or directory: '""/output'
Step #1 - "builder": File "/usr/local/bin/ftl.par/__main__/ftl/common/ftl_error.py", line 77, in InternalErrorHandler
Step #1 - "builder": File "/usr/local/bin/ftl.par/__main__.py", line 60, in main
Step #1 - "builder": File "/usr/local/bin/ftl.par/__main__.py", line 65, in <module>
Step #1 - "builder": exec code in run_globals
Step #1 - "builder": File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
Step #1 - "builder": "__main__", fname, loader, pkg_name)
Step #1 - "builder": File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
Step #1 - "builder": Traceback (most recent call last):
Step #1 - "builder": tar_runtime_package is likely not on the path
Step #1 - "builder": exited with error [Errno 12] Cannot allocate memory
Step #1 - "builder": ERROR tar_runtime_package tar -pcf /tmp/tmplWd4v8.tar --hard-dereference --transform flags=r;s,^,/.googleconfig/, --exclude *.pyc .
Step #1 - "builder": INFO full build took 79 seconds
Step #1 - "builder": INFO build process for FTL image took 79 seconds
Step #1 - "builder": INFO Building app layer took 0 seconds
Step #1 - "builder": INFO tar_runtime_package took 0 seconds
Step #1 - "builder": INFO tar_runtime_package tar -pcf /tmp/tmplWd4v8.tar --hard-dereference --transform flags=r;s,^,/.googleconfig/, --exclude *.pyc .
Step #1 - "builder": INFO starting: tar_runtime_package
Step #1 - "builder": INFO starting: Building app layer
Step #1 - "builder": INFO Building app layer took 76 seconds
Step #1 - "builder": INFO Finished gzipping tarfile.
Step #1 - "builder": INFO gzip_tar_runtime_package took 58 seconds
Step #1 - "builder":
Step #1 - "builder": INFO `gzip_tar_runtime_package` stdout:
Step #1 - "builder": INFO gzip_tar_runtime_package gzip /tmp/tmpStZGpR.tar -1
Step #1 - "builder": INFO starting: gzip_tar_runtime_package
Step #1 - "builder": INFO tar_runtime_package took 8 seconds
Step #1 - "builder":
Step #1 - "builder": INFO `tar_runtime_package` stdout:
Step #1 - "builder": INFO tar_runtime_package tar -pcf /tmp/tmpStZGpR.tar --hard-dereference --transform flags=r;s,^,/srv/, --exclude *.pyc .
Step #1 - "builder": INFO starting: tar_runtime_package
Step #1 - "builder": INFO starting: Building app layer
Step #1 - "builder": INFO checking_cached_requirements.txt_layer took 1 seconds
Step #1 - "builder": INFO [CACHE][HIT] v0.17.0:PYTHON (requirements)->62b99263456c1af3bff1307a585449789858bb17c695a2accfa55a3e5ee8f950
Step #1 - "builder": INFO check python version took 0 seconds
Step #1 - "builder":
Step #1 - "builder": INFO `python version` stderr:
Step #1 - "builder": /opt/python3.7/bin/python3.7 --version
Step #1 - "builder": INFO `python version` full cmd:
Step #1 - "builder": INFO starting: check python version
Step #1 - "builder":
Step #1 - "builder": gunicorn
Step #1 - "builder": tensorflow==2.0.0
Step #1 - "builder": gensim==3.8.1
Step #1 - "builder": scipy==1.3.2
Step #1 - "builder": numpy==1.17.4
Step #1 - "builder": Flask==1.1.1
Step #1 - "builder": INFO new_descriptor_contents:
Step #1 - "builder":
Step #1 - "builder": gunicorn
Step #1 - "builder": tensorflow==2.0.0
Step #1 - "builder": gensim==3.8.1
Step #1 - "builder": scipy==1.3.2
Step #1 - "builder": numpy==1.17.4
Step #1 - "builder": Flask==1.1.1
Step #1 - "builder": INFO descriptor_contents:
Step #1 - "builder": INFO {"architecture": "amd64", "author": "Unknown", "config": {}, "created": "2019-12-02T22:00:00Z", "history": [{"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Unknown", "created": "2019-12-02T22:00:00Z", "created_by": "//containerregistry/client:ftl.par"}, {"author": "Bazel", "created": "1970-01-01T00:00:00Z", "created_by": "bazel build ..."}], "os": "linux", "rootfs": {"diff_ids":
谢谢!
2个回答
感谢您的反馈,我终于让它工作了。我不得不将 app.yaml 更改为手动缩放和 env flex:
runtime: python37
env: flex
manual_scaling:
instances: 1
resources:
cpu: 2
memory_gb: 8
disk_size_gb: 10
并且我必须创建一个 docker 映像,并将其推送到存储库,然后使用此映像进行部署。它是 Dockerfile:
FROM gcr.io/google-appengine/python
COPY ["main.py", "/"]
COPY ["model.h5", "/"]
COPY ["run_server.sh", "/"]
RUN chmod +x /run_server.sh
RUN pip3 install --upgrade pip
RUN pip3 install tensorflow && \
pip3 install h5py && \
pip3 install flask && \
pip3 install gensim
EXPOSE 8080
ENTRYPOINT ["/run_server.sh"]
CMD ["master"]
这是我使用的 run_server.sh:
#!/bin/bash
export LC_ALL=C.UTF-8
export LANG=C.UTF-8
python3 -u /main.py
然后您必须遵循以下步骤:
Generation Docker Image: docker build . -t [IMAGE]
Tag Local Docker Image: docker tag [IMAGE] eu.gcr.io/[PROJECT-ID]/[IMAGE]
Push Docker Image: docker push eu.gcr.io/[PROJECT-ID]/[IMAGE]
Deploy App: gcloud app deploy --image-url=eu.gcr.io/[PROJECT-ID]/[IMAGE]
现在它运行良好 :)
Daniel T.
2019-12-04