llhuii
95f6432a0c
Makefile: add push for storage-initializer image
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
KubeEdge Bot
cb77bb10a2
Merge pull request #52 from llhuii/add-storage-initializer
Add storage-initializer code/Dockerfile
4 years ago
KubeEdge Bot
54779115db
Merge pull request #63 from llhuii/fix-issue-56
gm/incrementallearningjob.go: fix wrong kind
4 years ago
KubeEdge Bot
ac90f18596
Merge pull request #61 from JimmyYang20/main
Add shared storage code for LC
4 years ago
JimmyYang20
656520c093
Fix shared storage code in LC according the comments
Signed-off-by: JimmyYang20 <yangjin39@huawei.com>
4 years ago
llhuii
b631ff592a
Parallel downloading s3 objects in download.py
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
f14f36cd9e
gm: add the rbac of accessing secret/configmap
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
5e5bdb4c7a
gm: fix secret error log in downstream.go
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
e80f458989
Fix secrets of model/dataset in federated/joint
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
a4faed0220
gm/incrementallearningjob.go: fix wrong kind
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
2aeed1ca3d
Fix the issues of storage download script
1. fix the content of indirect url
2. s/S3_ENDPOINT/&_URL/
3. refactor the loggings
4. fix localpath bug
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
aba17a7dfc
Fix the logging content of the LC-message readable
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
5df522c67e
Update crd definition by 'make crds'
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
e7e7b2cba0
gm: add secret support for shared storage
1. add 'CredientalName' into dataset/model/incrementallearningjob
2. add the secret info to storage-initializer download container if
needed
3. add the secret info to worker container if needed
4. sync the secret infos to LC in downstream.go if needed
5. add indirect url support
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
JimmyYang20
f88f6b1d50
Add shared storage code for LC
Signed-off-by: JimmyYang20 <yangjin39@huawei.com>
4 years ago
llhuii
7466cc9dc4
Add indirect download for storage download script
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
JimmyYang20
747057e308
Update vendor
Signed-off-by: JimmyYang20 <yangjin39@huawei.com>
4 years ago
llhuii
fb483c6c6e
gm: add s3-like storage support with public access
1. add s3-like storage support
2. abstract storage-related(s3, hostpath) into storage_initializer_injector.go
3. refactor worker-related code into worker.go
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
1cf495f41b
Add storage-initializer code/Dockerfile
supported protocols:
1. http(s)
2. s3 compatible with the env AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY,
AWS_ENDPOINT_URL
Since the image kfserving/storage-initializer is big(~341MB) and have
some limits with http server, the end image size is 54.7MB:
1. support to download more than one uri
2. smaller image size: use minio-client instead of boto3(less than
40MB), remove gcs/azure
3. remove the http-server content-type limits
4. add obs protocol
5. also extract .tar.gz/.zip for s3
6. fix pep8 style
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
KubeEdge Bot
d3a647261a
Merge pull request #49 from llhuii/add_issue_template
Add two issue templates
4 years ago
KubeEdge Bot
51566c3376
Merge pull request #48 from llhuii/update-install-script
Update install script
4 years ago
llhuii
bb4c96414f
Optimize the install.sh
1. retry to curl yamls
2. add 'SEDNA_ROOT' to install from existing source directory
3. force delete for the case deletion hangs
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
78f85eaf8a
Add two issue templates
1. bug-report.md
2. enhancement.md
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
25bd966d1c
docs: update the installation doc
1. rm git prerequisite
2. explictly add 'SEDNA_ACTION=create' to install cmd
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
KubeEdge Bot
a643d6bb19
Merge pull request #45 from JimmyYang20/main
Fix example of federated learning
4 years ago
JimmyYang
1508c2e764
Fix example of federated learning
Signed-off-by: JimmyYang <yangjin39@huawei.com>
4 years ago
KubeEdge Bot
88250d2035
Merge pull request #43 from llhuii/add-installation-all-in-one-script
Add all-in-one installation script
4 years ago
llhuii
08bb8b4c5f
Add all-in-one installation script
1. add all-in-one installation script. also support uninstall operation.
2. update the installation docs using the all-in-one script
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
KubeEdge Bot
1941a6d013
Merge pull request #37 from TymonXie/main
docs: enhance example of incremental learning
4 years ago
KubeEdge Bot
0428d69259
Merge pull request #38 from JimmyYang20/main
Fix examples: joint inference/federated learning
4 years ago
JimmyYang
b0b7cb992d
Enhance federated learning example
Signed-off-by: JimmyYang <yangjin39@huawei.com>
4 years ago
JimmyYang
36e1df2314
Enhance joint inference example
Signed-off-by: JimmyYang <yangjin39@huawei.com>
4 years ago
TymonXie
ee71aedec8
docs: enhance example of incremental learning
Signed-off-by: TymonXie <tymonxie@gmail.com>
4 years ago
KubeEdge Bot
0ecda80b0a
Merge pull request #42 from llhuii/fix-incremental-hem
incremental-learning: fix hem name/parameters
4 years ago
llhuii
e5445dd8da
incremental-learning: fix hem name/parameters
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
KubeEdge Bot
bbd05d1bf3
Merge pull request #41 from llhuii/update-installation-doc
docs: update installation instructions
4 years ago
llhuii
ac0ca178e2
docs: update the installation instructions
1. add note about the node GM deployed at
2. remove build sections
3. squash GM's steps
4. fix some misleading sentences
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
KubeEdge Bot
32838c2861
Merge pull request #33 from llhuii/add-pod-template
Add pod template support
4 years ago
llhuii
e63d4a1e17
Makefile: add image push targets
1. add 'push' to push gm/lc images
2. add 'push-examples' to push examples images
3. add 'push-all' for both
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
d832d48c66
Makefile: fix IMAGE_REPO/IMAGE_TAG's default value
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
23a4da0c63
gm: optimize workerParam code
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
14e384b849
incremental-learning: fix dataset relative path
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
7d09d000f9
Update proposals with pod-template
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
904a7c7197
Fix image repo in local-up script
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
e8400b4fe4
kubectl-create crds instead of kubectl-apply
since we switch to pod template which has large crd yaml, this would
lead to the error `metadata.annotations: Too long: must have at most
262144 bytes` when `kubectl apply -f`
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
c569f8a341
docs: update install.md
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
0fc9bffe76
Make the name of examples images more meaningful
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
5e2aa14b3e
examples: add Dockerfiles and build script
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
83a2e132e6
Update crd samples with pod-template
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago
llhuii
940ec7049c
Add pod template for incremental learning
Signed-off-by: llhuii <liulinghui@huawei.com>
4 years ago