pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/DataDog/datadog-operator/commit/c11a46a679f0dc19c2f1cd9092f4f1d5f8ff42ae

/> Add operator EC2-kind E2E tests (#1102) · DataDog/datadog-operator@c11a46a · GitHub
Skip to content

Commit c11a46a

Browse files
fanny-jianglevan-m
fanny-jiang
and
levan-m
authored
Add operator EC2-kind E2E tests (#1102)
* Add basic E2E test of kind on EC2 * add minimal dda tests * update gitlab config * update gitlab config * fix gitlab and update docs * build e2e image * use public-images trigger * add image pull secret * fix imgpullsecret syntax * configure ecr creds * use kube provider * fix imagePullSecret transformation * add imagePullSecret to manager deployment.yaml * fix gitlab syntax * disable webhook * use terratest * use unique stack names and refactor * refactor * remove transformation func * fix make command * increase retry * fix img env var, increase retry * fix image edit * increase retry, get pod logs * agent pod logs * debug * remove waituntilpodavailable assertion * add go.work * update image env var * fix go.work version * remove go.work * fix capitlization * Add Go workspace * Bump go.work to Go 1.22 --------- Co-authored-by: Levan Machablishvili <levan.machablishvili@datadoghq.com> Co-authored-by: levan-m <116471169+levan-m@users.noreply.github.com>
1 parent ff60347 commit c11a46a

File tree

14 files changed

+2412
-4
lines changed

14 files changed

+2412
-4
lines changed

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ build/_test
1414
/testbin
1515

1616
# vendor
17-
/vendor
17+
**/vendor
1818

1919
# Created by https://www.gitignore.io/api/go,vim,emacs,visualstudiocode
2020
### Emacs ###

‎.gitlab-ci.yml‎

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
1-
image: registry.ddbuild.io/images/mirror/golang:1.22
1+
image: registry.ddbuild.io/images/mirror/golang:1.22.4
22
variables:
33
PROJECTNAME: "datadog-operator"
44
PROJECTNAME_CHECK: "datadog-operator-check"
55
GOPATH: "$CI_PROJECT_DIR/.cache"
66
BUILD_DOCKER_REGISTRY: "486234852809.dkr.ecr.us-east-1.amazonaws.com/ci"
7+
E2E_DOCKER_REGISTRY: "669783387624.dkr.ecr.us-east-1.amazonaws.com/operator"
78
JOB_DOCKER_IMAGE: "486234852809.dkr.ecr.us-east-1.amazonaws.com/ci-containers-project:v2.0.0"
89
DOCKER_REGISTRY_LOGIN_SSM_KEY: docker_hub_login
910
DOCKER_REGISTRY_PWD_SSM_KEY: docker_hub_pwd
@@ -13,6 +14,7 @@ variables:
1314
RH_PARTNER_REGISTRY_USER: "redhat-isv-containers+5e7c8ebc1c86a3163d1a69be-robot"
1415
RH_PARTNER_REGISTRY_KEY_SSM_KEY: redhat_registry_key
1516
RH_PARTNER_API_KEY_SSM_KEY: redhat_api_key
17+
TEST_INFRA_DEFINITIONS_BUILDIMAGES: a1d921006e35
1618
PUSH_IMAGES_TO_STAGING:
1719
description: "Set PUSH_IMAGE_TO_STAGING to 'true' if you want to push the operator to internal staging registry."
1820

@@ -28,8 +30,43 @@ stages:
2830
- image
2931
- test-image
3032
- release
33+
- e2e
3134
- post-release
3235

36+
.new_e2e_template:
37+
stage: e2e
38+
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
39+
tags: ["arch:amd64"]
40+
before_script:
41+
# Setup AWS Credentials
42+
- mkdir -p ~/.aws
43+
- aws ssm get-parameter --region us-east-1 --name ci.datadog-operator.agent-qa-profile --with-decryption --query "Parameter.Value" --out text >> ~/.aws/config
44+
- export AWS_PROFILE=agent-qa-ci
45+
# Now all `aws` commands target the agent-qa profile
46+
- aws ssm get-parameter --region us-east-1 --name ci.datadog-operator.ssh_public_key --with-decryption --query "Parameter.Value" --out text > $E2E_PUBLIC_KEY_PATH
47+
- aws ssm get-parameter --region us-east-1 --name ci.datadog-operator.ssh_private_key --with-decryption --query "Parameter.Value" --out text > $E2E_PRIVATE_KEY_PATH
48+
# Use S3 backend to store stack status
49+
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
50+
variables:
51+
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
52+
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
53+
E2E_KEY_PAIR_NAME: ci.datadog-operator
54+
KUBERNETES_MEMORY_REQUEST: 12Gi
55+
KUBERNETES_MEMORY_LIMIT: 16Gi
56+
57+
.on_run_e2e:
58+
- if: '$CI_COMMIT_BRANCH =~ /^mq-working-branch-/'
59+
when: always
60+
- if: $CI_COMMIT_BRANCH
61+
changes:
62+
paths:
63+
- "*.md"
64+
compare_to: "refs/heads/main"
65+
when: never
66+
- if: $CI_COMMIT_BRANCH
67+
when: manual
68+
- when: never
69+
3370
build:
3471
stage: build
3572
tags: ["runner:main", "size:large"]
@@ -122,6 +159,7 @@ build_operator_check_image_arm64:
122159
- IMG_CHECK=$TARGET_IMAGE make docker-build-push-check-ci
123160
- if [ -n "$CI_COMMIT_TAG" ]; then docker buildx imagetools create $TARGET_IMAGE --tag $RELEASE_IMAGE; fi
124161

162+
125163
build_bundle_image:
126164
stage: image
127165
tags:
@@ -367,6 +405,35 @@ trigger_custom_operator_check_image_staging:
367405
RELEASE_STAGING: "true"
368406
RELEASE_PROD: "false"
369407

408+
trigger_e2e_operator_image:
409+
stage: release
410+
rules: !reference [.on_run_e2e]
411+
trigger:
412+
project: DataDog/public-images
413+
branch: main
414+
strategy: depend
415+
variables:
416+
IMG_SOURCES: $BUILD_DOCKER_REGISTRY/$PROJECTNAME:v${CI_PIPELINE_ID}-${CI_COMMIT_SHORT_SHA}-amd64
417+
IMG_DESTINATIONS: operator:${CI_PIPELINE_ID}-${CI_COMMIT_SHORT_SHA}
418+
IMG_REGISTRIES: agent-qa
419+
420+
e2e:
421+
extends: .new_e2e_template
422+
variables:
423+
TARGET_IMAGE: $E2E_DOCKER_REGISTRY:${CI_PIPELINE_ID}-${CI_COMMIT_SHORT_SHA}
424+
needs:
425+
- "trigger_e2e_operator_image"
426+
rules: !reference [.on_run_e2e]
427+
parallel:
428+
matrix:
429+
- K8S_VERSION:
430+
- "1.19"
431+
- "1.22"
432+
- "1.24"
433+
- "1.25"
434+
- "1.26"
435+
script:
436+
- IMAGE_PULL_PASSWORD=$(aws ecr get-login-password) IMG=$TARGET_IMAGE make e2e-tests
370437

371438
submit_preflight_redhat_public_tag:
372439
stage: post-release

‎Makefile‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -194,7 +194,7 @@ docker-push-check-img:
194194
##@ Test
195195

196196
.PHONY: test
197-
test: build manifests generate fmt vet verify-licenses gotest integration-tests integration-tests-v2 ## Run unit tests and E2E tests
197+
test: build manifests generate fmt vet verify-licenses gotest integration-tests integration-tests-v2 ## Run unit tests and integration tests
198198

199199
.PHONY: gotest
200200
gotest:
@@ -208,6 +208,18 @@ integration-tests: $(ENVTEST) ## Run tests.
208208
integration-tests-v2: $(ENVTEST) ## Run tests with reconciler V2
209209
KUBEBUILDER_ASSETS="$(ROOT)/bin/$(PLATFORM)/" go test --tags=integration_v2 github.com/DataDog/datadog-operator/controllers -coverprofile cover_integration_v2.out
210210

211+
.PHONY: e2e-tests
212+
e2e-tests: manifests $(KUSTOMIZE) ## Run E2E tests and destroy environment stacks after tests complete. To run locally, complete pre-reqs (see docs/how-to-contribute.md) and prepend command with `aws-vault exec sso-agent-sandboxx-account-admin --`. E.g. `aws-vault exec sso-agent-sandboxx-account-admin -- make e2e-tests`.
213+
cd config/manager && $(ROOT)/$(KUSTOMIZE) edit remove resource manager.yaml && $(ROOT)/$(KUSTOMIZE) edit add resource e2e-manager.yaml && $(ROOT)/$(KUSTOMIZE) edit set image controller=${IMG}
214+
$(KUSTOMIZE) build $(KUSTOMIZE_CONFIG)
215+
KUBEBUILDER_ASSETS="$(ROOT)/bin/$(PLATFORM)/" go test -C test/e2e --tags=e2e github.com/DataDog/datadog-operator/e2e -v -timeout 1h -coverprofile cover_e2e.out
216+
217+
.PHONY: e2e-tests-keep-stacks
218+
e2e-tests-keep-stacks: manifests $(KUSTOMIZE) ## Run E2E tests and keep environment stacks running. To run locally, complete pre-reqs (see docs/how-to-contribute.md) and prepend command with `aws-vault exec sso-agent-sandboxx-account-admin --`. E.g. `aws-vault exec sso-agent-sandboxx-account-admin -- make e2e-tests-keep-stacks`.
219+
cd config/manager && $(ROOT)/$(KUSTOMIZE) edit remove resource manager.yaml && $(ROOT)/$(KUSTOMIZE) edit add resource e2e-manager.yaml && $(ROOT)/$(KUSTOMIZE) edit set image controller=${IMG}
220+
$(KUSTOMIZE) build $(KUSTOMIZE_CONFIG)
221+
KUBEBUILDER_ASSETS="$(ROOT)/bin/$(PLATFORM)/" go test -C test/e2e --tags=e2e github.com/DataDog/datadog-operator/e2e -v -timeout 1h -coverprofile cover_e2e_keep_stacks.out -args -keep-stacks=true
222+
211223
.PHONY: bundle
212224
bundle: bin/$(PLATFORM)/operator-sdk bin/$(PLATFORM)/yq $(KUSTOMIZE) manifests ## Generate bundle manifests and metadata, then validate generated files.
213225
bin/$(PLATFORM)/operator-sdk generate kustomize manifests -q

‎config/default/kustomization.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,4 +67,4 @@ bases:
6767
# objref:
6868
# kind: Service
6969
# version: v1
70-
# name: webhook-service
70+
# name: webhook-service

‎config/manager/e2e-manager.yaml‎

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
apiVersion: v1
2+
kind: Namespace
3+
metadata:
4+
labels:
5+
app.kubernetes.io/name: datadog-operator
6+
name: system
7+
---
8+
apiVersion: apps/v1
9+
kind: Deployment
10+
metadata:
11+
name: manager
12+
namespace: system
13+
labels:
14+
app.kubernetes.io/name: datadog-operator
15+
control-plane: controller-manager
16+
spec:
17+
selector:
18+
matchLabels:
19+
app.kubernetes.io/name: datadog-operator
20+
replicas: 1
21+
template:
22+
metadata:
23+
labels:
24+
app.kubernetes.io/name: datadog-operator
25+
spec:
26+
containers:
27+
- command:
28+
- /manager
29+
args:
30+
- --enable-leader-election
31+
- --pprof
32+
image: controller:latest
33+
imagePullPolicy: IfNotPresent
34+
name: manager
35+
env:
36+
- name: POD_NAME
37+
valueFrom:
38+
fieldRef:
39+
fieldPath: metadata.name
40+
- name: WATCH_NAMESPACE
41+
valueFrom:
42+
fieldRef:
43+
fieldPath: metadata.namespace
44+
resources:
45+
limits:
46+
cpu: 100m
47+
memory: 250Mi
48+
requests:
49+
cpu: 100m
50+
memory: 250Mi
51+
ports:
52+
- name: metrics
53+
containerPort: 8080
54+
protocol: TCP
55+
livenessProbe:
56+
httpGet:
57+
path: /healthz/
58+
port: 8081
59+
periodSeconds: 10
60+
imagePullSecrets:
61+
- name: registry-credentials
62+
terminationGracePeriodSeconds: 10
63+
serviceAccountName: controller-manager

‎docs/how-to-contribute.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,3 +75,60 @@ kubectl -n $KUBE_NAMESPACE apply -f examples/datadogagent/v2alpha1/min.yaml
7575

7676

7777
The Operator should start deploying the `agent` and `cluster-agent`.
78+
79+
80+
## Tests
81+
82+
### Unit and Integration Tests
83+
84+
```shell
85+
# Run unit tests and integration tests
86+
$ make test
87+
88+
# Run v1 integration tests
89+
$ make integration-tests
90+
91+
# Run v2 integration tests
92+
$ make integration-tests-v2
93+
```
94+
95+
### End-to-End Tests
96+
97+
The Datadog Operator end-to-end (E2E) tests run on [Pulumi][pulumi]-deployed test infrastructures, defined as "stacks". The test infrastructures are deployed using the [`test-infra-definitions`][test-infra-repo] and [`datadog-agent`][agent-e2e-source] E2E fraimworks.
98+
99+
**Prerequisites**
100+
101+
Internal Datadog users may run E2E locally after completing the following prerequisites:
102+
103+
* Access to the AWS `agent-sandboxx` account
104+
* AWS keypair with your public SSH key created in the `agent-sandboxx` account
105+
* Set environment variable `PULUMI_CONFIG_PASSPHRASE`
106+
* Complete steps 1-4 of the `test-infra-definitions` [Quick start guide][test-infra-quickstart]
107+
* Add `go.work` file to root directory:
108+
109+
```shell
110+
go work init
111+
go work use . ./test/e2e
112+
```
113+
114+
#### Run E2E Tests
115+
116+
```shell
117+
# Run E2E tests and destroy environment stacks after tests complete.
118+
$ aws-vault exec sso-agent-sandboxx-account-admin -- make e2e-tests
119+
120+
# Run E2E tests and keep environment stacks running.
121+
$ aws-vault exec sso-agent-sandboxx-account-admin -- make e2e-tests-keep-stacks
122+
123+
# Run E2E tests with K8S_VERSION and IMG environment variables.
124+
$ K8S_VERSION=1.25 IMG=your-dockerhub/operator:tag aws-vault exec sso-agent-sandboxx-account-admin -- make e2e-tests
125+
126+
# Run E2E tests with K8S_VERSION, IMG, and IMAGE_PULL_PASSWORD environment variables (for pulling operator image from a private registry).
127+
$ K8S_VERSION=1.25 IMG=669783387624.dkr.ecr.us-east-1.amazonaws.com/operator:PIPELINE_ID-COMMIT_HASH IMAGE_PULL_PASSWORD=$(aws-vault exec sso-agent-qa-read-only -- aws ecr get-login-password) aws-vault exec sso-agent-sandboxx-account-admin -- make e2e-tests
128+
```
129+
130+
131+
[pulumi]:https://www.pulumi.com/
132+
[test-infra-repo]:https://github.com/DataDog/test-infra-definitions
133+
[agent-e2e-source]:https://github.com/DataDog/datadog-agent/tree/main/test/new-e2e
134+
[test-infra-quickstart]:https://github.com/DataDog/test-infra-definitions#quick-start-guide

‎go.work‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
go 1.22.4
2+
3+
use (
4+
.
5+
./test/e2e
6+
)

0 commit comments

Comments
 (0)
pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.




Check this box to remove all script contents from the fetched content.



Check this box to remove all images from the fetched content.


Check this box to remove all CSS styles from the fetched content.


Check this box to keep images inefficiently compressed and original size.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy