ユニファ開発者ブログ

ユニファ株式会社システム開発部メンバーによるブログです。

Datadog Meetupに参加してきました!けど記事の内容はTerraform v0.12絡み!

おはこんばんちは!

インフラ担当のすずきです!

2019年初ブログでしょうか?(記憶が曖昧

正月明け頃から咳がではじめて、今週頭くらいまでずっと悩まされていました。
後述するイベントまでにはなんとか収まってホッとしております。

昨晩DatadogのMeetupイベント「Datadogはじめました! - connpass」に参加してLTもしてまいりました。
咳が収まってなかったらただでさえ拙い発表が、ひどいものになっていたのではないか…!!(人生2回目のLT)

さて今回のブログの内容ですが、参加した内容は割愛して私がLTで発表したTerraform v0.12 とそれに対応したdatadog-providerを使った話をもう少し詳しく書こうかと思います。

発表資料は末尾に記載しておくのでご興味あるかたは御覧ください。

Terraform v0.12 のインストール

とりあえず今普通に手に入るのはv0.11系ですね
でもとりあえずtfenv見てみましょう(みなさんtfenv使ってます?便利ですよ)

❯ tfenv list-remote
0.12.0
0.11.11
0.11.10
0.11.9
0.11.9-beta1

あるじゃん!!

❯ tfenv install 0.12.0
[INFO] Installing Terraform v0.12.0
[INFO] Downloading release tarball from https://releases.hashicorp.com/terraform/0.12.0/terraform_0.12.0_darwin_amd64.zip

curl: (22) The requested URL returned error: 403
tfenv: tfenv-install: [ERROR] Tarball download failed

うっ…そんなうまくはいかないですね

Releases · hashicorp/terraform · GitHub こちらみるとv0.12のalpha1までならあるぞ…

Tags · hashicorp/terraform · GitHub タグだとalpha4まである…しかたないビルドしよう!!

ほぼほぼ公式のREADME.mdどおりに行いましょう

❯ go get github.com/hashicorp/terraform
❯ cd "$GOPATH/src/github.com/hashicorp/terraform"

ディレクトリに移動してブランチをalpha4に変更します。

❯ git checkout v0.12.0-alpha4

引き続き公式のとおりに

❯ make tools
GO111MODULE=off go get -u github.com/kardianos/govendor
GO111MODULE=off go get -u golang.org/x/tools/cmd/stringer
GO111MODULE=off go get -u golang.org/x/tools/cmd/cover
GO111MODULE=off go get -u github.com/golang/mock/mockgen
❯ make
==> Checking that code complies with gofmt requirements...
./terraform/resource_provisioner.go:70:25: expected type, found '='
./state/state.go:24:12: expected type, found '='
./state/state.go:27:18: expected type, found '='
./state/state.go:30:18: expected type, found '='
./state/state.go:33:21: expected type, found '='
./state/state.go:36:21: expected type, found '='
./state/state.go:39:13: expected type, found '='
./state/state.go:114:15: expected type, found '='
./state/state.go:117:16: expected type, found '='
./helper/schema/schema.go:375:18: expected type, found '='
gofmt needs running on the following files:
./terraform/context_apply_test.go
./plugin/discovery/get_test.go
./backend/remote-state/s3/backend_test.go
./backend/remote-state/azure/backend_test.go
./backend/remote-state/azure/client_test.go
./states/resource_test.go
./command/meta_backend_test.go
./command/init.go
./helper/schema/schema_test.go
./helper/schema/shims_test.go
./helper/schema/resource_timeout_test.go
./builtin/provisioners/chef/linux_provisioner_test.go
You can use the command: `make fmt` to reformat code.
make: *** [fmtcheck] Error 1

くっ…なんだこれは…、Goのバージョンかもしれない今のバージョンを確認します

❯ go version
go version go1.8.3 darwin/amd64

公式を確認して… f:id:mominosin:20190123172841p:plain

なるほど完全に理解した

❯ brew upgrade go
〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜
〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜〜
==> Pouring go-1.11.4.mojave.bottle.1.tar.gz
🍺  /usr/local/Cellar/go/1.11.4: 9,298 files, 404.3MB

いざ尋常に…

❯ make
==> Checking that code complies with gofmt requirements...
# We turn off modules for "go generate" because our downstream generate
# commands are not all ready to deal with Go modules yet, and this
# avoids downloading all of the deps that are in the vendor dir anyway.
GO111MODULE=off go generate ./...
2019/01/22 11:34:47 Generated command/internal_plugin_list.go
GO111MODULE=off go fmt command/internal_plugin_list.go > /dev/null
go list -mod=vendor ./... | xargs -t -n4 go test  -mod=vendor -timeout=2m -parallel=4
build flag -mod=vendor only valid when using modules

成功したっぽいですね、あとはmake binでバイナリ作って

❯ XC_OS="darwin" XC_ARCH="amd64" make bin
==> Checking that code complies with gofmt requirements...
# We turn off modules for "go generate" because our downstream generate
# commands are not all ready to deal with Go modules yet, and this
# avoids downloading all of the deps that are in the vendor dir anyway.
GO111MODULE=off go generate ./...
2019/01/22 11:46:29 Generated command/internal_plugin_list.go
GO111MODULE=off go fmt command/internal_plugin_list.go > /dev/null
==> Removing old directory...
go: modules disabled inside GOPATH/src by GO111MODULE=auto; see 'go help modules'
==> Building...
Number of parallel builds: 3

-->    darwin/amd64: github.com/hashicorp/terraform
==> Packaging...
--> darwin_amd64
  adding: terraform (deflated 79%)

==> Results:
total 232256
-rwxr-xr-x  1 shingo.suzuki  staff   113M  1 22 11:50 terraform
❯ ./bin/terraform version
Terraform v0.12.0

これで、terraform v0.12のビルドができました。
Terraformだけv0.12になってもprovider側も対応させないといけません。

Datadog Providerのv0.12対応版をビルド

Branches · terraform-providers/terraform-provider-datadog · GitHub Datadogのproviderのブランチを見てみると、ありました! v0.12-upgrade-2018-12-13 これをビルドしてみましょう。

これもREADME.md通りに

❯ mkdir -p $GOPATH/src/github.com/terraform-providers
❯ cd $GOPATH/src/github.com/terraform-providers
❯ git clone git@github.com:terraform-providers/terraform-provider-datadog
❯ cd terraform-provider-datadog

ブランチを変更して

❯ git checkout v0.12-upgrade-2018-12-13

そいやっ

❯ make build

==> Checking that code complies with gofmt requirements...
go install

とりあえず成功したみたい、実体はこちら

❯ ls $GOPATH/bin/terraform-provider-datadog

ビルドしたプロバイダーを Terraformが読み取ってくれるところに配置

手でビルドしたプロバイダーどうやって認識させるんだろうなと思ってたら、Datadog ProviderのREADME.mdに公式へのリンクがあったのでそちらを参照 Plugin Basics - Terraform by HashiCorp

これに書いてあるとおりに配置してみる

❯ mkdir -p ~/.terraform.d/plugins
❯ cp $GOPATH/bin/terraform-provider-datadog ~/.terraform.d/plugins

プロバイダーを使ってみる

早速素振りしてみましょう

❯ mkdir datadog-terraform12 ; cd datadog-terraform12

以下のファイルを配置

## Terraformの変数あとで環境変数から読み込ませます。
variable "datadog_api_key" {
  default = ""
}

variable "datadog_app_key" {
  default = ""
}

provider "datadog" {
  api_key = "${var.datadog_api_key}"
  app_key = "${var.datadog_app_key}"
}

動かしてみる

❯ export TF_VAR_datadog_app_key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
❯ export TF_VAR_datadog_api_key=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
### terraform 0.12どうやって叩くか悩んだので手元に…
❯ cp $GOPATH/src/github.com/hashicorp/terraform/bin/terraform .
❯ ./terraform init

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

外部から取得した際はここにプロバイダーが入ってるはずなのですが存在してないのでローカルに存在してるの使ってそうですね!

❯ ls .terraform/plugins/darwin_amd64/
lock.json

v0.11のときのファイルをplanしてみる

サンプルで11のときのdashboardをplanしてみます。ファイルはこちら datadog-terraform-v0.12/dashboard.tf.error at master · mominosin/datadog-terraform-v0.12 · GitHub

❯ ./terraform plan

Error: Unsupported block type

  on /Users/shingo.suzuki/Work/git/datadog-terraform12/dashboard.tf line 14, in resource "datadog_timeboard" "sample_12":
  14:       style {

Blocks of type "style" are not expected here. Did you mean to define argument
"style"? If so, use the equals sign to assign it a value.


Error: Unsupported block type

  on /Users/shingo.suzuki/Work/git/datadog-terraform12/dashboard.tf line 29, in resource "datadog_timeboard" "sample_12":
  29:       style {

Blocks of type "style" are not expected here. Did you mean to define argument
"style"? If so, use the equals sign to assign it a value.


Error: Unsupported block type

  on /Users/shingo.suzuki/Work/git/datadog-terraform12/dashboard.tf line 45, in resource "datadog_timeboard" "sample_12":
  45:       style {

Blocks of type "style" are not expected here. Did you mean to define argument
"style"? If so, use the equals sign to assign it a value.


Error: Unsupported block type

  on /Users/shingo.suzuki/Work/git/datadog-terraform12/dashboard.tf line 60, in resource "datadog_timeboard" "sample_12":
  60:       style {

Blocks of type "style" are not expected here. Did you mean to define argument
"style"? If so, use the equals sign to assign it a value.

めっちゃ怒られました。
ブロック内に1つだけしか定義できないものは = つけないとだめだったようなので style の箇所を修正し、再度 plan してみます。 ファイルはこちら datadog-terraform-v0.12/dashboard.tf at master · mominosin/datadog-terraform-v0.12 · GitHub

❯ ./terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # datadog_timeboard.sample_11 will be created
  + resource "datadog_timeboard" "sample_12" {
      + description = "created using the Datadog provider in Terraform"
      + id          = (known after apply)
      + read_only   = true
      + title       = "Terraform v0.12"

      + graph {
          + precision = "0"
          + title     = "ALB Response Time 01"
          + viz       = "timeseries"

          + request {
              + q       = "avg:aws.applicationelb.target_response_time.average{*}"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "solid"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
      + graph {
          + precision = "0"
          + title     = "ALB Response Time 02"
          + viz       = "timeseries"

          + request {
              + q       = "week_before(avg:aws.applicationelb.target_response_time.average{*})"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "dotted"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
      + graph {
          + precision = "0"
          + title     = "ALB Response Time 03"
          + viz       = "timeseries"

          + request {
              + q       = "avg:aws.applicationelb.target_response_time.average{*}"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "solid"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
      + graph {
          + precision = "0"
          + title     = "ALB Response Time 04"
          + viz       = "timeseries"

          + request {
              + q       = "week_before(avg:aws.applicationelb.target_response_time.average{*})"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "dotted"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

動きました。planの結果もなんかきれいですね!!

v0.12の新機能 for_eachを使って比較してみる

先程のtfファイルを、for_eachで書き換えてみました。ファイルはこちら datadog-terraform-v0.12/dashboard_for_each.tf at master · mominosin/datadog-terraform-v0.12 · GitHub
graphブロックを1つだけでループで複製されるようにしています。
ただ、本設定は(https://www.hashicorp.com/blog/hashicorp-terraform-0-12-preview-for-and-for-each) 公式のブログの記事を参考に書いてるのですが、そこのfor_eachの書き方だとエラーになってしまったので違った書き方をしています、ご了承ください。

そしてplan

❯ ./terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.


------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # datadog_timeboard.sample_12_for_each will be created
  + resource "datadog_timeboard" "sample_12_for_each" {
      + description = "created using the Datadog provider in Terraform"
      + id          = (known after apply)
      + read_only   = true
      + title       = "Terraform v0.12 for_each"

      + graph {
          + precision = "0"
          + title     = "ALB Response Time 01"
          + viz       = "timeseries"

          + request {
              + q       = "avg:aws.applicationelb.target_response_time.average{*}"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "solid"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
      + graph {
          + precision = "0"
          + title     = "ALB Response Time 02"
          + viz       = "timeseries"

          + request {
              + q       = "week_before(avg:aws.applicationelb.target_response_time.average{*})"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "dotted"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
      + graph {
          + precision = "0"
          + title     = "ALB Response Time 03"
          + viz       = "timeseries"

          + request {
              + q       = "avg:aws.applicationelb.target_response_time.average{*}"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "solid"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
      + graph {
          + precision = "0"
          + title     = "ALB Response Time 04"
          + viz       = "timeseries"

          + request {
              + q       = "week_before(avg:aws.applicationelb.target_response_time.average{*})"
              + stacked = false
              + style   = {
                  + "palette" = "orange"
                  + "type"    = "dotted"
                  + "width"   = "normal"
                }
              + type    = "line"
            }
        }
    }

Plan: 1 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

どうでしょう?同じような結果になってないですかね?
Terraform V0.11以前の count をつかった疑似ループではこのようにはいきません。

おわり

ながながとTerraformビルド結果や出力、providerの設定ばかり貼り付けてしまったので見づらかったら申し訳ないです。
まだ出る気配のないv0.12ですが出たときにすぐ移行できるようにこのブログ参考にお試しいただけたら幸いです。

おまけ: イベントの発表資料