windwos环境和pyspark环境;多git配置问题解决;github相关
一: 打不开cmd/cmd闪退问题解决
1.win+R打开 regedit
2.在注册表找到 HKEY_CURRENT_USER\Software\Microsoft\Command Processor路径
3.在该路径下找到AutoRun,将数值改为空
可能原因:找到AutoRun 发现值是: if exist & if exist "D:\software\miniforge3\condabin\conda_hook.bat" "D:\software\miniforge3\condabin\conda_hook.bat"
可能是版权问题删除anaconda后安装miniforge3默认导致的。
二:pyspark环境打包问题解决:
pysaprk需要python3环境,conda建环境,python39环境,报如下错误:
"/data08/hadoop/yarn/local/usercache/dms/appcache/application_1719854044076_229309/container_e217_1719854044076_229309_02_000001/pyspark.zip/pyspark/cloudpickle.py", line 127, in _make_cell_set_template_code TypeError: an integer is required (got type bytes)
网上搜是版本问题,python版本或者pyspark版本。
新建环境,python37环境,解决了。
三:pip安装包各种问题
docx包:
pip install docs -i https://pypi.tuna.tsinghua.edu.cn/simple/
运行 报错:
ModuleNotFoundError Traceback (most recent call last)
Cell In[8], line 1
----> 1 import docx
2 import os
3 import json
File d:\software\miniforge3\envs\tf1.14_py39\lib\site-packages\docx.py:30
27 except ImportError:
28 TAGS = {}
---> 30 from exceptions import PendingDeprecationWarning
31 from warnings import warn
33 import logging
ModuleNotFoundError: No module named 'exceptions
解决办法:
pip uninstall docx
pip install python-docx -i https://pypi.tuna.tsinghua.edu.cn/simple/
即可。
四:windows同时安装gitlab和github多个git
一个用来公司用。一个用来自己的demo用。
Windows上同时配置GitHub,Gitlab,Gitee(码云)三种开发环境_window搭建码云类似的-CSDN博客
https://blog.csdn.net/kyyius/article/details/116232389
参考以上2篇文章。
补充点:
1.链接1提到全局配置要删除,亲测不用删除也可以,github和gitlab都可以正常提交任务。
2.关键是.ssh/下新建confit文件。配置不同的github和gitlab的选择。
3.默认是.ssh/路径下生产私钥和公钥,如果换个新的名字,在哪个路径下生成,秘钥就落到哪个路径下。
五:github相关
1.访问限制
有时候国内打不开github,加上kk前缀即可快速打开。
kkgithub.com/langgenius/dify
2.vscode文本编辑器在线打开
github1s.com
github.dev
六:python web
1.flask
我部署的flask服务,别人调用http接口请求我的服务,我在我的请求里接收参数,再请求别人的requests.post 调用另一个的服务2次。本地启动flask,流程都是通的,但是部署到docker里如果是请求参数是短文本,都正常,如果是长文本,则会报错。
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 134, in handle
self.handle_request(listener, req, client, addr)
File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/sync.py", line 177, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1498, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1473, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 880, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 865, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "/app/app.py", line 138, in ai_writing
response = requests.post(ai_writing_url, json=req_params_map, headers=headers,timeout=600)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 724, in send
history = [resp for resp in gen]
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 724, in <listcomp>
history = [resp for resp in gen]
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 265, in resolve_redirects
resp = self.send(
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 536, in _make_request
response = conn.getresponse()
File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 464, in getresponse
httplib_response = super().getresponse()
File "/usr/local/lib/python3.9/http/client.py", line 1377, in getresponse
response.begin()
File "/usr/local/lib/python3.9/http/client.py", line 320, in begin
version, status, reason = self._read_status()
File "/usr/local/lib/python3.9/http/client.py", line 281, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/local/lib/python3.9/socket.py", line 704, in readinto
return self._sock.recv_into(b)
File "/usr/local/lib/python3.9/site-packages/gunicorn/workers/base.py", line 204, in handle_abort
sys.exit(1)
发现是 下面这行命令报错。
# 启动容器时运行的命令
CMD gunicorn -w 4 --bind 0.0.0.0:5000 app:app
gunicorn是多进程启动。这是启动 Gunicorn 服务器的命令,
-
-w 6:-w或--workers选项后面跟的数字表示工作进程的数量。在这个例子中,6表示 Gunicorn 将启动 6 个工作进程来处理传入的请求。工作进程的数量通常取决于服务器的 CPU 核心数和预期的负载。
本地启动没问题,线上docker里部署短文本请求没问题。长文本单次requests.post没问题。但是我这里有2次的requests.post 就报上面的问题了。
解决办法:
dockerfile 里这样启动,不用多进程了。
CMD ["sh", "-c", "python app.py"]
可能的原因是的多线程并发这块,flask的天然不太好。
看到这句话:
```
如果你追求高性能、异步编程和类型安全,FastAPI可能是更好的选择。如果你注重简洁性、灵活性和成熟的生态系统,Flask是一个可靠的选择
```
更多推荐
所有评论(0)