Commit Graph

10402 Commits

Author SHA1 Message Date
danfengliu
fc9d37e0d3 Refind build base process in Makefile
Remove build base executable in Makefile by replacing it as an input parameter.
Add add more input parameters for controlling docker pull/push to make
build base process flexible for users.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-06-08 17:40:28 +08:00
stonezdj(Daojun Zhang)
e579545f89
Merge pull request #15025 from stonezdj/2.2_fall_back_local
(cherry-pick) Fall back to local registry when upstream registry is not working
2021-06-04 11:12:21 +08:00
ChenYu Zhang
4b018ce953
Merge pull request #14977 from chlins/perf/cherry-pick-artifacts
(cherry-pick)Improve the performance of artifact related APIs
2021-06-02 10:33:58 +08:00
Wenkai Yin
ba90b9c30e Improve the performance of artifact related APIs
Improve the performance of artifact related APIs by adding indexes and refactoring sql logic

Closes #13890 #14813 #14814

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-06-01 11:04:14 +08:00
ChenYu Zhang
3b65db5190
fix: change sql to improve the performance of catalog api (#14976)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2021-06-01 10:40:41 +08:00
stonezdj
0d2c501bf8 Fall back to local registry when upstream registry is not working
Fixes #14822
When upstream registry not working, but status might stay healthy because the health check interval is 5 minutes, if a pull request comes before registry status turns to unhealthy, the proxy cache middleware might proxy the request to the upstream registry and get a 401 error and this 401 error might translate to a http 500 error to the client eventually.

To solve this issue, it fall back all error to local registry when proxying manifest except the NotFoundError from the local registry.

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-06-01 10:07:39 +08:00
Wenkai Yin(尹文开)
e0a8dcee25
Merge pull request #15016 from reasonerjt/merge-slash-middleware-v2.2
[Cherrypick - 2.2]: Add merge slash middleware
2021-06-01 09:15:34 +08:00
Wenkai Yin(尹文开)
b333bff9ee
Merge pull request #15018 from reasonerjt/v2auth-enhancement-v2.2
[Cherrypick - v2.2]: Make v2auth more strict
2021-06-01 09:14:53 +08:00
Daniel Jiang
13a4edada4 Make v2auth more strict
This commit enhances the v2auth middleware, such that any un-recognized
request sent to /v2/ will be blocked.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-31 15:39:58 +08:00
Daniel Jiang
0308b63bb5 Add merge slash middleware
There are code in the core component to conditionally execute code based
on the pattern of url path, and different ingress controller or reverse
proxy may handle the dup slashes in the url path differently.
This commit merge dup slashes in the url paths to make things more
consistent.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-05-31 13:43:29 +08:00
Wenkai Yin(尹文开)
8b71e0bd07
Merge pull request #14828 from kofj/fix/2.2.0-tcr-provider
Fix tencentcr adapter.
2021-05-20 14:01:10 +08:00
danfengliu
56d7937fd3
Merge pull request #14868 from danfengliu/cherry-pick-2.2-replace-email-domain-name-in-test
Cherry pick 2.2:replace email domain name in test
2021-05-15 17:11:46 +08:00
danfengliu
1b016e1834 Cherry pick 2.2:replace email domain name in test
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-05-15 07:39:42 +00:00
danfengliu
9fd780156f
Merge pull request #14885 from wy65701436/release-2.2.2
bump up base to v2.2.2
2021-05-14 18:32:48 +08:00
Wang Yan
053a518812 Merge branch 'release-2.2.0' of https://github.com/goharbor/harbor into release-2.2.2 2021-05-14 17:31:39 +08:00
He Weiwei
7052d3d91b
[Cherry pick]perf: cache the metadata of the scanner (#14883)
perf: cache the metadata of the scanner

1. Cache the metadata of scanner 30s.
2. Change the scanner client request timeout to 5s.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-14 17:15:23 +08:00
He Weiwei
ac30941406
fix: using new ctx to scan artifact when scanAll (#14884)
DB transaction failure may break the loop query of the artifacts, the
result is that not all artifacts are scanned in one scan all job. Using
a new DB connection to call the Scan method of the controller to avoid
this problem.

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-14 15:49:41 +08:00
stonezdj(Daojun Zhang)
76af3c2699
Merge pull request #14867 from stonezdj/2.2_add_check_usergroup_api
Check permission for user group api
2021-05-14 15:32:37 +08:00
Wang Yan
ee76d69947 bump up base to v2.2.2
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-05-14 15:09:17 +08:00
stonezdj
d32e13df30 Check permission for user group api
Check create/update permission on rbac.ResourceUserGroup

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-05-13 20:43:16 +08:00
Wenkai Yin(尹文开)
dc0e10de49
Merge pull request #14852 from ywk253100/210511_scheduler
Fix duplicate execution record issue
2021-05-12 15:58:59 +08:00
stonezdj(Daojun Zhang)
9a10911691
Merge pull request #14724 from reasonerjt/authproxy-admin-username-v2.2
[Cherrypick to v2.2]: Add attribute admin username for authproxy
2021-05-12 12:29:46 +08:00
He Weiwei
5163bc3fe8
fix: change art_size to bigint in migrations (#14842)
Signed-off-by: He Weiwei <hweiwei@vmware.com>
2021-05-12 08:39:40 +08:00
Wenkai Yin
d06120c663 Fix duplicate execution record issue
When the core service cannot response the checkin request in time, duplicated execution records may be created, this commit introduces the revision column to make sure there is only one record for one schedule trigger

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-05-11 21:19:36 +08:00
fanjiankong
f2c08a95b7 Fix tencentcr adapter.
1. Limit API qps for the adapter.
2. Allow set qps via env.
3. Fix Tencnet SDK pagenation.
4. Fix resource filter.

Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2021-05-10 11:41:52 +08:00
Qian Deng
27f7f5e95e
Merge pull request #14700 from ninjadq/move_python_flask_to_system_2_2
Fix: Move pyyaml to system package
2021-04-29 13:18:45 +08:00
Wenkai Yin(尹文开)
9b98d18e09
Merge pull request #14755 from ywk253100/210427_replicate_chart
Fix the failure of event based chart replication
2021-04-29 09:51:32 +08:00
Wenkai Yin(尹文开)
0b7a991fdf
Merge pull request #14761 from ywk253100/210427_task_not_found
Read upstream job ID as ID when it isn't null
2021-04-29 09:50:04 +08:00
Wenkai Yin
f13c98c346 Read upstream job ID as ID when it isn't null
Read upstream job ID as ID when it isn't null to fix the job not found error

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-04-27 14:45:33 +08:00
Wenkai Yin
be37c937ec Fix the failure of event based chart replication
Fix the failure of event based chart replication

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2021-04-27 09:32:39 +08:00
danfengliu
1ea87f14e2
Merge pull request #14570 from danfengliu/cherry-pick-optimize-e2e-engine-image-size
[Cherry Pick 2.2.0] Add manifest and CNAB replication tests
2021-04-25 18:18:34 +08:00
danfengliu
3dae9491e0 Add manifest and CNAB replication tests
1. Add manifest and CNAB replication tests;
2. Duplicate ORAS and sigularity tests from API test to nightly common
tests;
3. Optimize get dns code in CI;
4. Optimize E2E dockerfile;
5. Sample image size should be cover requirement for large size like 512M.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-04-25 16:41:58 +08:00
danfengliu
e955f51498
Merge pull request #14670 from danfengliu/cherry-pick-2.2-add-build-base-image-job-in-ci
[ Cherry-pick 2.2.0 ] Add build base image step in build package git action workflow
2021-04-23 15:46:42 +08:00
danfengliu
958af4d3d6 Fix build base related issues
1. Add build base image step in build package git action workflow;
2. Add build base step to UT test in CI, base image used by UI test should be built before building harbor image in the same runtime;
3. In build package workflow, trigger build base image step in condition of changing both in
Dockerfile.base and VERSION;
4. Add tag for setup nightly test.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-04-22 17:39:33 +08:00
Daniel Jiang
b52a844a79
Refine the "not support" error when searching user (#14695)
If authenticator does not support searching user/group, wraps it as a
not found error, such that the API will return 404 rather than confusing
500.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-04-21 14:10:07 +08:00
Daniel Jiang
3c39c7d2bc Add attribute admin username for authproxy
This commit adds the attribute "http_authproxy_admin_usernames", which
is string that contains usernames separated by comma, when a user logs
in and the username in the tokenreview status matches the setting of
this attribute, the user will have administrator permission.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-04-21 09:56:36 +08:00
DQ
402a088af0 Fix: Move pyyaml to system package
pothon version of pyyaml will solve the cve

Signed-off-by: DQ <dengq@vmware.com>
2021-04-20 19:45:47 +08:00
Steven Zou
29916d162f
Merge pull request #14680 from bitsf/fix_aws_checkRepository_2.2
[cherry-pick] fix(replication) aws adapter checkRepository 2.2.2
2021-04-19 11:18:01 +08:00
Ziming Zhang
bc14740bbb fix(replication) aws adapter checkRepository
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2021-04-17 15:46:01 +08:00
Steven Zou
44b83257ca
Merge pull request #14647 from steven-zou/fix/value_out_of_range_2.2
CHERRY-PICK: fix(jobservice): value out of range (#14634)
2021-04-14 18:51:48 +08:00
Archer1A
d2e1f629a1 fix(jobservice): value out of range (#14634)
value out of range when job count bigger than 256

Signed-off-by: vic <985695123@qq.com>
2021-04-14 14:35:36 +08:00
Will Sun
a2d0f7fe9a
Support proxy cache for docker-registry type (#14632)
Signed-off-by: AllForNothing <sshijun@vmware.com>
2021-04-14 13:43:14 +08:00
Wang Yan
0c64adf9ec
fix immutable issue (#14644)
Fixes #14568

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-04-14 13:25:32 +08:00
stonezdj
b9bbf94c31 Support proxy cache for docker-registry type
Add proxy cache for docker registry type
Fixes #14477, #14547
Signed-off-by: stonezdj <stonezdj@gmail.com>

(cherry picked from commit ccd9ee8c56)
2021-04-14 10:58:14 +08:00
Qian Deng
07ab364851
Merge pull request #14626 from ninjadq/fix_python_cve_2_2
Fix python cve 2 2
2021-04-14 10:36:18 +08:00
Steven Zou
cf756802b0
Merge pull request #14629 from steven-zou/fix/job_log_not_found_issue_2.2
fix(js):job log not found issue
2021-04-13 14:15:21 +08:00
Steven Zou
3a691dccc3 fix(js):job log not found issue
- return 404 code when the job log is not found

Signed-off-by: Steven Zou <szou@vmware.com>
2021-04-13 12:55:12 +08:00
dependabot[bot]
800753dd86 Bump pyyaml from 4.2b1 to 5.4 in /make/photon/prepare
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 4.2b1 to 5.4.
- [Release notes](https://github.com/yaml/pyyaml/releases)
- [Changelog](https://github.com/yaml/pyyaml/blob/master/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/commits/5.4)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-13 10:48:00 +08:00
dependabot[bot]
3cb3c59d41 Bump jinja2 from 2.11.1 to 2.11.3 in /make/photon/prepare
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.1 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.1...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-13 10:46:01 +08:00
Steven Zou
562d4e693c
Merge pull request #14602 from steven-zou/fix/cherry-resending-2.2
fix(js): webhook infinitely resending issue
2021-04-12 10:29:23 +08:00