Dynamodb

DynamoDB 是一个可扩展的 AWS 托管的 NoSQL 数据库。 它支持 key-value 和文档数据模型,使您的数据能够有一个灵活的 schema 。 此扩展提供的功能允许运行在 Quarkus 的客户端与服务端(DynamoDB) 通信。 您可以在 Amazon DynamoDB 网站找到更多有关 DynamoDB 的信息。

DynamoDB 扩展基于 AWS Java SDK 2.x。 它在 1.x 代码基础的重写提供了两种编程模型(阻塞式 & 异步方式)。 请记住它正在积极开发,SDK 1.x 中不是所有功能都支持,例如 Document APIsObject Mappers

This technology is considered 预览.

For a full list of possible extension statuses, check our FAQ entry.

此 Quarkus 扩展支持两种编程模式:

  • 阻塞式使用 URL Connection HTTP client (默认) 或 Apache HTTP Client

  • 异步编程模式 基于 JDK 的 CompletableFuture 对象和 Netty HTTP client。

In this guide, we see how you can get your REST services to use the DynamoDB locally and on AWS.

必备条件

要完成本指南,您需要:

  • 安装了JDK 1.8+ 并正确配置了 JAVA_HOME

  • IDE

  • Apache Maven 3.6.2+

  • 访问 DynamoDB 服务的 AWS 帐户

  • 可选,Docker 用来本地运行 DynamoDB 进行测试

本地安装 DynamoDB

使用 DynamoDB 最简单方式是通过容器运行一个本地实例。

docker run --publish 8000:8000 amazon/dynamodb-local:1.11.477 -jar DynamoDBLocal.jar -inMemory -sharedDb

这将启动一个 DynamoDB 实例,可通过端口 8000 访问。 您可以通过访问 http://localhost:8000/shell 上的 web shell 来检查其运行情况。

查看 本地安装 DynamoDB 指南 了解运行 DynamoDB 的其他选项。

在您的浏览器中打开 http://localhost:8000/shell

复制并粘贴以下代码到shell并运行它:

var params = {
    TableName: 'QuarkusFruits',
    KeySchema: [{ AttributeName: 'fruitName', KeyType: 'HASH' }],
    AttributeDefinitions: [{  AttributeName: 'fruitName', AttributeType: 'S', }],
    ProvisionedThroughput: { ReadCapacityUnits: 1, WriteCapacityUnits: 1, }
};

dynamodb.createTable(params, function(err, data) {
    if (err) ppJson(err);
    else ppJson(data);

});

在 AWS 上安装 DynamoDB

在您使用 DynamoDB 的 AWS SDKs 之前,您必须获得一个 AWS access key ID 和 secret access key。 欲了解更多信息,请查阅https://docs.aws.amazon.com/amazondynameodbraft/developerguide/SettingUp.DynamoWebService.html[安装 DynamoDB (Web Service)]。

我们建议使用 AWS CLI 创建表(table):

aws dynamodb create-table --table-name QuarkusFruits \
                          --attribute-definitions AttributeName=fruitName,AttributeType=S \
                          --key-schema AttributeName=fruitName,KeyType=HASH \
                          --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

成果

此应用程序管理存储在 Amazon DynamoDB 中的元素 (fruits) 。

我们建议您按照下面章节指示,一步一步创建应用程序。 但是你也可以直接跳到已完成的例子。

克隆 Git 仓库: git clone https://github.com/quarkusio/quarkus-quickstarts.git, 或下载 压缩包

代码在 dynamodb-quickstart 目录 中。

创建 Maven 项目

首先,我们需要一个新项目。 通过以下命令创建一个新项目:

mvn io.quarkus:quarkus-maven-plugin:1.3.1.Final:create \
    -DprojectGroupId=org.acme \
    -DprojectArtifactId=dynamodb-quickstart \
    -DclassName="org.acme.dynamodb.FruitResource" \
    -Dpath="/fruits" \
    -Dextensions="resteasy-jsonb,dynamodb,resteasy-mutiny"
cd dynamodb-quickstart

此命令生成一个 Maven 项目结构并导入 RESTEasy/JAX-RS, Mutiny 及 DynamoDB Client extensions。 然后 amazon-dynamodb 的扩展也添加到你的 pom.xml 以及 RESTEasy 的 Mutiny 支持。

创建 JSON REST 服务

在这个例子中,我们将创建一个应用程序来管理一系列的 fruits 。 示例应用程序将展示由扩展支持的两种编程模式。

首先,让我们创建 Fruit bean 如下所示:

package org.acme.dynamodb;

import java.util.Map;
import java.util.Objects;

import io.quarkus.runtime.annotations.RegisterForReflection;
import software.amazon.awssdk.services.dynamodb.model.AttributeValue;

@RegisterForReflection
public class Fruit {

    private String name;
    private String description;

    public Fruit() {
    }

    public static Fruit from(Map<String, AttributeValue> item) {
        Fruit fruit = new Fruit();
        if (item != null && !item.isEmpty()) {
            fruit.setName(item.get(AbstractService.FRUIT_NAME_COL).s());
            fruit.setDescription(item.get(AbstractService.FRUIT_DESC_COL).s());
        }
        return fruit;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getDescription() {
        return description;
    }

    public void setDescription(String description) {
        this.description = description;
    }

    @Override
    public boolean equals(Object obj) {
        if (!(obj instanceof Fruit)) {
            return false;
        }

        Fruit other = (Fruit) obj;

        return Objects.equals(other.name, this.name);
    }

    @Override
    public int hashCode() {
        return Objects.hash(this.name);
    }
}

没有什么特别。 需要注意的一个重要问题是,JSON 序列化层需要一个默认构造函数。 静态的 from 方法用 DynamoDB 客户端得到的 Map 对象创建了一个 bean 。

现在创建一个 org.acme.dynamodb.AbstractService ,它将由辅助方法组成,用来准备 DynamoDB 读取对象和添加数据到 table 中。

package org.acme.dynamodb;

import java.util.HashMap;
import java.util.Map;

import software.amazon.awssdk.services.dynamodb.model.AttributeValue;
import software.amazon.awssdk.services.dynamodb.model.GetItemRequest;
import software.amazon.awssdk.services.dynamodb.model.PutItemRequest;
import software.amazon.awssdk.services.dynamodb.model.ScanRequest;

public abstract class AbstractService {

    public final static String FRUIT_NAME_COL = "fruitName";
    public final static String FRUIT_DESC_COL = "fruitDescription";

    public String getTableName() {
        return "QuarkusFruits";
    }

    protected ScanRequest scanRequest() {
        return ScanRequest.builder().tableName(getTableName())
                .attributesToGet(FRUIT_NAME_COL, FRUIT_DESC_COL).build();
    }

    protected PutItemRequest putRequest(Fruit fruit) {
        Map<String, AttributeValue> item = new HashMap<>();
        item.put(FRUIT_NAME_COL, AttributeValue.builder().s(fruit.getName()).build());
        item.put(FRUIT_DESC_COL, AttributeValue.builder().s(fruit.getDescription()).build());

        return PutItemRequest.builder()
                .tableName(getTableName())
                .item(item)
                .build();
    }

    protected GetItemRequest getRequest(String name) {
        Map<String, AttributeValue> key = new HashMap<>();
        key.put(FRUIT_NAME_COL, AttributeValue.builder().s(name).build());

        return GetItemRequest.builder()
                .tableName(getTableName())
                .key(key)
                .attributesToGet(FRUIT_NAME_COL, FRUIT_DESC_COL)
                .build();
    }
}

然后,创建一个 org.acme.dynamodb.FruitSyncService 作为我们应用程序的业务层,并使用同步模式 client 存取 DynamoDB 的 fruits 。

package org.acme.dynamodb;

import java.util.List;
import java.util.stream.Collectors;

import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;

import software.amazon.awssdk.services.dynamodb.DynamoDbClient;

@ApplicationScoped
public class FruitSyncService extends AbstractService {

    @Inject
    DynamoDbClient dynamoDB;

    public List<Fruit> findAll() {
        return dynamoDB.scanPaginator(scanRequest()).items().stream()
                .map(Fruit::from)
                .collect(Collectors.toList());
    }

    public List<Fruit> add(Fruit fruit) {
        dynamoDB.putItem(putRequest(fruit));
        return findAll();
    }

    public Fruit get(String name) {
        return Fruit.from(dynamoDB.getItem(getRequest(name)).item());
    }
}

现在,编辑 org.acme.dynamodb.FruitResource 类:

package org.acme.dynamodb;

import java.util.List;

import javax.inject.Inject;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

@Path("/fruits")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class FruitResource {

    @Inject
    FruitSyncService service;

    @GET
    public List<Fruit> getAll() {
        return service.findAll();
    }

    @GET
    @Path("{name}")
    public Fruit getSingle(@PathParam("name") String name) {
        return service.get(name);
    }

    @POST
    public List<Fruit> add(Fruit fruit) {
        service.add(fruit);
        return getAll();
    }
}

实现非常简单,您只需要使用 JAX-RS 批注来定义您的接口,并使用 FruitSyncService 来列出/添加新的 fruits 。

配置 DynamoDB 客户端

DynamoDB 客户端 (同步和 异步) 都可以通过 src/main/resources 目录中 application.properties 文件配置。 此外,您需要将同步模式的实现添加到 classpath。 默认情况下,扩展使用 URL connection HTTP client,所以 您需要将 URL connection client 依赖添加到 pom.xml 文件:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>url-connection-client</artifactId>
</dependency>

如果您想要使用 Apache HTTP 客户端,其配置如下:

quarkus.dynamodb.sync-client.type=apache

然后添加以下依赖到应用程序的 pom.xml

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>apache-client</artifactId>
    <exclusions>
        <exclusion>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
        </exclusion>
    </exclusions>
</dependency>
重要的是从依赖中排除 commons-logging 以强制 Apache HTTP client 使用 Quarkus logger。

如果您要使用本地的 DynamoDB 实例,请将其配置如下:

quarkus.dynamodb.endpoint-override=http://localhost:8000

quarkus.dynamodb.aws.region=eu-central-1
quarkus.dynamodb.aws.credentials.type=static
quarkus.dynamodb.aws.credentials.static-provider.access-key-id=test-key
quarkus.dynamodb.aws.credentials.static-provider.secret-access-key=test-secret
  • quarkus.dynamodb.aws.region - 它是客户端所需要的,但由于您正在使用本地DynamoDB实例,因此可以选择任何有效的 AWS 区域。

  • quarkus.dynamodb.aws.credentials.type - 设为 static 凭据提供者,包含 access-key-id and secret-access-key

  • quarkus.dynamodb.endpoint-override - 指示 DynamoDB 客户端使用本地实例而不是AWS 服务

如果您想要使用 AWS 帐户,您需要设置它:

quarkus.dynamodb.aws.region=<YOUR_REGION>
quarkus.dynamodb.aws.credentials.type=default
  • quarkus.dynamodb.aws.region 你应该将它设置到你提供DynamoDB table 的region。

  • quarkus.dynamodb.aws.credentials.type - 使用 default 按照下列顺序来寻找凭据:

  • Java System Properties - aws.accessKeyIdaws.secretKey

    • 环境变量 - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

    • 所有AWS SDKs 和 AWS CLI 共享的默认凭据文件 (~/.aws/credentials)

    • Credentials delivered through the Amazon EC2 container service if the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and the security manager has permission to access the variable,

    • Instance profile credentials delivered through the Amazon EC2 metadata service

后续步骤

打包

打包应用程序简单运行下 ./mvnw clean package 就可以。 它可以通过 java -jar target/dynamodb-quickstart-1.0-SNAPSHOT-runner.jar 运行。

通过 GraalVM 安装,您也可以创建一个本机可执行文件: ./mvnw clean package -Dnative 。 这将需要一些时间,取决于您的系统。

异步模式

感谢 Quarkus extension 使用的 AWS SDK v2.x ,您可以开箱即用异步模式编程。

创建一个 org.acme.dynamodb.FruitAsyncService ,它类似于我们的 FruitSyncService 但使用异步编程模式。

package org.acme.dynamodb;

import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.stream.Collectors;

import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;

import io.smallrye.mutiny.Uni;
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient;

@ApplicationScoped
public class FruitAsyncService extends AbstractService {

    @Inject
    DynamoDbAsyncClient dynamoDB;

    public Uni<List<Fruit>> findAll() {
        return Uni.createFrom().completionStage(() -> dynamoDB.scan(scanRequest()))
                .onItem().apply(res -> res.items().stream().map(Fruit::from).collect(Collectors.toList()));
    }

    public Uni<List<Fruit>> add(Fruit fruit) {
        return Uni.createFrom().completionStage(() -> dynamoDB.putItem(putRequest(fruit)))
                .onItem().ignore().andSwitchTo(this::findAll);
    }

    public Uni<Fruit> get(String name) {
        return Uni.createFrom().completionStage(() -> dynamoDB.getItem(getRequest(name)))
                .onItem().apply(resp -> Fruit.from(resp.item()));
    }
}

前边代码中,我们从异步 DynamoDB 客户端返回的 CompltionStage 对象中创建 Uni 实例然后转化已产生的数据。

然后,创建调用此异步服务的异步 REST 接口:

package org.acme.dynamodb;

import io.smallrye.mutiny.Uni;

import javax.inject.Inject;
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;
import java.util.List;

@Path("/async-fruits")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class FruitAsyncResource {

    @Inject
    FruitAsyncService service;

    @GET
    public Uni<List<Fruit>> getAll() {
        return service.findAll();
    }

    @GET
    @Path("{name}")
    public Uni<Fruit> getSingle(@PathParam("name") String name) {
        return service.get(name);
    }

    @POST
    public Uni<List<Fruit>> add(Fruit fruit) {
        return service.add(fruit)
                .onItem().ignore().andSwitchTo(this::getAll);
    }
}

并将 Netty HTTP 客户端依赖添加到 pom.xml

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>netty-nio-client</artifactId>
</dependency>

配置参考

Configuration property fixed at build time - All other configuration properties are overridable at runtime

Configuration property

Type

Default

List of execution interceptors that will have access to read and modify the request and response objects as they are processed by the AWS SDK. The list should consists of class names which implements software.amazon.awssdk.core.interceptor.ExecutionInterceptor interface.

list of class name

Type of the sync HTTP client implementation

url, apache

url

Enable DynamoDB service endpoint discovery.

boolean

false

The endpoint URI with which the SDK should communicate. If not specified, an appropriate endpoint to be used for DynamoDB service and region.

URI

The amount of time to allow the client to complete the execution of an API call. This timeout covers the entire client execution except for marshalling. This includes request handler execution, all HTTP requests including retries, unmarshalling, etc. This value should always be positive, if present.

Duration

The amount of time to wait for the HTTP request to complete before giving up and timing out. This value should always be positive, if present.

Duration

An Amazon Web Services region that hosts DynamoDB.

It overrides region provider chain with static value of region with which the DynamoDB client should communicate.

If not set, region is retrieved via the default providers chain in the following order:

  • aws.region system property

  • region property from the profile file

  • Instance profile file

See software.amazon.awssdk.regions.Region for available regions.

Region

Configure the credentials provider that should be used to authenticate with AWS.

Available values:

  • default - the provider will attempt to identify the credentials automatically using the following checks:

    • Java System Properties - aws.accessKeyId and aws.secretKey

    • Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

    • Credential profiles file at the default location (~/.aws/credentials) shared by all AWS SDKs and the AWS CLI

    • Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable is set and security manager has permission to access the variable.

    • Instance profile credentials delivered through the Amazon EC2 metadata service

  • static - the provider that uses the access key and secret access key specified in the static-provider section of the config.

  • system-property - it loads credentials from the aws.accessKeyId, aws.secretAccessKey and aws.sessionToken system properties.

  • env-variable - it loads credentials from the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN environment variables.

  • profile - credentials are based on AWS configuration profiles. This loads credentials from a profile file, allowing you to share multiple sets of AWS security credentials between different tools like the AWS SDK for Java and the AWS CLI.

  • container - It loads credentials from a local metadata service. Containers currently supported by the AWS SDK are Amazon Elastic Container Service (ECS) and AWS Greengrass

  • instance-profile - It loads credentials from the Amazon EC2 Instance Metadata Service.

  • process - Credentials are loaded from an external process. This is used to support the credential_process setting in the profile credentials file. See Sourcing Credentials From External Processes for more information.

  • anonymous - It always returns anonymous AWS credentials. Anonymous AWS credentials result in un-authenticated requests and will fail unless the resource or API’s policy has been configured to specifically allow anonymous access.

default, static, system-property, env-variable, profile, container, instance-profile, process, anonymous

default

Whether this provider should fetch credentials asynchronously in the background. If this is true, threads are less likely to block, but additional resources are used to maintain the provider.

boolean

false

Whether the provider should reuse the last successful credentials provider in the chain. Reusing the last successful credentials provider will typically return credentials faster than searching through the chain.

boolean

true

string

string

The name of the profile that should be used by this credentials provider. If not specified, the value in AWS_PROFILE environment variable or aws.profile system property is used and defaults to default name.

string

Whether the provider should fetch credentials asynchronously in the background. If this is true, threads are less likely to block when credentials are loaded, but additional resources are used to maintain the provider.

boolean

false

The amount of time between when the credentials expire and when the credentials should start to be refreshed. This allows the credentials to be refreshed before they are reported to expire.

Duration

15S

The maximum size of the output that can be returned by the external process before an exception is raised.

MemorySize

1024

The command that should be executed to retrieve credentials.

string

The maximum amount of time to establish a connection before timing out.

Duration

2S

The amount of time to wait for data to be transferred over an established, open connection before the connection is timed out.

Duration

30S

The amount of time to wait when acquiring a connection from the pool before giving up and timing out.

Duration

10S

The maximum amount of time that a connection should be allowed to remain open while idle.

Duration

60S

The maximum amount of time that a connection should be allowed to remain open, regardless of usage frequency.

Duration

The maximum number of connections allowed in the connection pool. Each built HTTP client has its own private connection pool.

int

50

Whether the client should send an HTTP expect-continue handshake before each request.

boolean

true

Whether the idle connections in the connection pool should be closed asynchronously. When enabled, connections left idling for longer than quarkus.dynamodb.sync-client.connection-max-idle-time will be closed. This will not close connections currently in use.

boolean

true

boolean

false

The endpoint of the proxy server that the SDK should connect through. Currently, the endpoint is limited to a host and port. Any other URI components will result in an exception being raised.

URI

The username to use when connecting through a proxy.

string

The password to use when connecting through a proxy.

string

For NTLM proxies - the Windows domain name to use when authenticating with the proxy.

string

For NTLM proxies - the Windows workstation name to use when authenticating with the proxy.

string

Whether to attempt to authenticate preemptively against the proxy server using basic authentication.

boolean

The hosts that the client is allowed to access without going through the proxy.

list of string

TLS managers provider type.

Available providers:

  • none - Use this provider if you don’t want the client to present any certificates to the remote TLS host.

  • system-property - Provider checks the standard javax.net.ssl.keyStore, javax.net.ssl.keyStorePassword, and javax.net.ssl.keyStoreType properties defined by the JSSE.

  • file-store - Provider that loads a the key store from a file.

none, system-property, file-store

system-property

The maximum number of allowed concurrent requests. For HTTP/1.1 this is the same as max connections. For HTTP/2 the number of connections that will be used depends on the max streams allowed per connection.

int

50

The maximum number of pending acquires allowed. Once this exceeds, acquire tries will be failed.

int

10000

The amount of time to wait for a read on a socket before an exception is thrown. Specify 0 to disable.

Duration

30S

The amount of time to wait for a write on a socket before an exception is thrown. Specify 0 to disable.

Duration

30S

The amount of time to wait when initially establishing a connection before giving up and timing out.

Duration

10S

The amount of time to wait when acquiring a connection from the pool before giving up and timing out.

Duration

2S

The maximum amount of time that a connection should be allowed to remain open, regardless of usage frequency.

Duration

The maximum amount of time that a connection should be allowed to remain open while idle. Currently has no effect if quarkus.dynamodb.async-client.use-idle-connection-reaper is false.

Duration

60S

Whether the idle connections in the connection pool should be closed. When enabled, connections left idling for longer than quarkus.dynamodb.async-client.connection-max-idle-time will be closed. This will not close connections currently in use.

boolean

true

The HTTP protocol to use.

http1-1, http2

http1-1

The maximum number of concurrent streams for an HTTP/2 connection. This setting is only respected when the HTTP/2 protocol is used. 0 means unlimited.

int

0

The SSL Provider to be used in the Netty client. Default is OPENSSL if available, JDK otherwise.

jdk, openssl, openssl-refcnt

boolean

false

The endpoint of the proxy server that the SDK should connect through. Currently, the endpoint is limited to a host and port. Any other URI components will result in an exception being raised.

URI

The hosts that the client is allowed to access without going through the proxy.

list of string

TLS managers provider type.

Available providers:

  • none - Use this provider if you don’t want the client to present any certificates to the remote TLS host.

  • system-property - Provider checks the standard javax.net.ssl.keyStore, javax.net.ssl.keyStorePassword, and javax.net.ssl.keyStoreType properties defined by the JSSE.

  • file-store - Provider that loads a the key store from a file.

none, system-property, file-store

system-property

Enable the custom configuration of the Netty event loop group.

boolean

false

Number of threads to use for the event loop group. If not set, the default Netty thread count is used (which is double the number of available processors unless the io.netty.eventLoopThreads system property is set.

int

The thread name prefix for threads created by this thread factory used by event loop group. The prefix will be appended with a number unique to the thread factory and a number unique to the thread. If not specified it defaults to aws-java-sdk-NettyEventLoop

string

Configuration of the file store provider This configuration section is optional

Type

Default

path

required

Key store type. See the KeyStore section in the Java Cryptography Architecture Standard Algorithm Name Documentation for information about standard keystore types.

string

required

string

required

path

required

Key store type. See the KeyStore section in the Java Cryptography Architecture Standard Algorithm Name Documentation for information about standard keystore types.

string

required

string

required

About the Duration format

The format for durations uses the standard java.time.Duration format. You can learn more about it in the Duration#parse() javadoc.

You can also provide duration values starting with a number. In this case, if the value consists only of a number, the converter treats the value as seconds. Otherwise, PT is implicitly prepended to the value to obtain a standard java.time.Duration format.

About the MemorySize format

A size configuration option recognises string in this format (shown as a regular expression): [0-9]+[KkMmGgTtPpEeZzYy]?. If no suffix is given, assume bytes.

quarkus.pro 是基于 quarkus.io 的非官方中文翻译站 ,最后更新 2020/04 。
沪ICP备19006215号-8
QQ交流群:1055930959
微信群: