AWS導入(3)

2020年11月29日

ひととおり、できたのでlambda に落として見ようかなと
AWS
にも書いてあるし、多分かんたんに作れるだろうと

lambda関数書いてみます。

def lambda_handler(event, context):
    try:
        logger = logging.getLogger()
        logger.setLevel(logging.INFO)
        session = boto3.session.Session()
        table = boto3.resource('dynamodb', endpoint_url = "http://dynamodb:8000").Table('Quote')
        json_item = '{"feed_datetime":"20201001125959101", "sym":"BTC", "ask1":100.5, "ask1_size":2.0}'
        item = json.loads(json_item, parse_float=decimal.Decimal)
        table.put_item(
            Item=item
        )
    
        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'DONE!'
            }),
        }

    except Exception as e:
        logger.exception(e)
        return {
            'statusCode': 500,
            'body': json.dumps({
                'error_message': str(e)
            }),
        }

def lambda_handler(event, context):

try:

logger = logging.getLogger()

logger.setLevel(logging.INFO)

session = boto3.session.Session()

table = boto3.resource('dynamodb', endpoint_url = "http://dynamodb:8000").Table('Quote')

json_item = '{"feed_datetime":"20201001125959101", "sym":"BTC", "ask1":100.5, "ask1_size":2.0}'

item = json.loads(json_item, parse_float=decimal.Decimal)

table.put_item(

Item=item

)

return {

'statusCode': 200,

'body': json.dumps({

'message': 'DONE!'

}),

}

except Exception as e:

logger.exception(e)

return {

'statusCode': 500,

'body': json.dumps({

'error_message': str(e)

}),

}

こちらのような旧式の方法もあるのですが、現在はlayerという共通関数を噛ますことができるので適用してみます。
適用すれば、こんな感じで一発でbuildできます

$ sam build sam build FeedHandleLayer

1	$ sam build sam build FeedHandleLayer

ResourceにFeedHandlerLayerが追加します。
metadataのところのpythonのバージョン大事です

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.7
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /hello
            Method: get

  FeedAgentFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: feed_handler/
      Handler: app.lambda_handler
      Runtime: python3.7
      Events:
        HelloWorld:
          Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
          Properties:
            Path: /feed
            Method: get
      Layers:
      - !Ref FeedHandleLayer

  FeedHandleLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      Description: FeedHandleLayer
      ContentUri: 'layers'
      CompatibleRuntimes:
        - python3.7
    Metadata:
      BuildMethod: python3.7

Resources:

HelloWorldFunction:

Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction

Properties:

CodeUri: hello_world/

Handler: app.lambda_handler

Runtime: python3.7

Events:

HelloWorld:

Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api

Properties:

Path: /hello

Method: get

FeedAgentFunction:

Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction

Properties:

CodeUri: feed_handler/

Handler: app.lambda_handler

Runtime: python3.7

Events:

HelloWorld:

Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api

Properties:

Path: /feed

Method: get

Layers:

- !Ref FeedHandleLayer

FeedHandleLayer:

Type: AWS::Serverless::LayerVersion

Properties:

Description: FeedHandleLayer

ContentUri: 'layers'

CompatibleRuntimes:

- python3.7

Metadata:

BuildMethod: python3.7

もともとパッケージ配下にあった↑みたいなtemplete.yamlをlayer上に移動します
requirement.txtも同じlayer上に移します
参考
レイヤーの中にあるものはパッケージと同階層にいるように使えます

ビルドします。

$ sam build sam build FeedHandleLayer

1	$ sam build sam build FeedHandleLayer

ビルドすると、.aws-samのフォルダが作成されて、自分のパッケージとrequirementに書いたパッケージがすべてコピーされて置かれます。(つまり同階層にすべてある)
あと、ビルドと行っても全部コピーなので、直接弄ってもOK

api起動してみます

で、

sam local start-api --docker-network lambda-local

1	sam local start-api --docker-network lambda-local

アクセスしてみます。

[ERROR] Runtime.ImportModuleError: Unable to import module 'app': Unable to import required dependencies:
numpy:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.7 from "/var/lang/bin/python3.7"
  * The NumPy version is: "1.19.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

[ERROR] Runtime.ImportModuleError: Unable to import module 'app': Unable to import required dependencies:

numpy:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for

many reasons, often due to issues with your setup or how NumPy was

installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

* The Python version is: Python3.7 from "/var/lang/bin/python3.7"

* The NumPy version is: "1.19.4"

and make sure that they are the versions you expect.

Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

ここでワチャワチャ言っている現象が起きちゃいました。結局numpyの中身が違うっぽいです。

原因はpythonのversionがちがうことAND　macとliunxのC++のコンパイラーが違うせい？だと思う？

これが自分で

{"message": "3.7.8 (default, Aug  5 2020, 09:09:18) \n[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]\nLinux-4.19.76-linuxkit-x86_64-with-glibc2.2.5"}

1	{"message": "3.7.8 (default, Aug 5 2020, 09:09:18) \n[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)]\nLinux-4.19.76-linuxkit-x86_64-with-glibc2.2.5"}

これがAWS

(py37b) Macico:~ macico$ python
Python 3.7.6 (default, Mar 15 2020, 16:55:31)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin

(py37b) Macico:~ macico$ python

Python 3.7.6 (default, Mar 15 2020, 16:55:31)

[Clang 11.0.0 (clang-1100.0.33.8)] on darwin

つまり、3.7.6にしなければ行けないらしいですね。環境変えるのだるいのでdocker使います。

ただ

ここにもある通り、dockerでpython3.7はむずいっぽいです。めんどいので3.6に変更
その場合は、3.6にlayerの設定を変更します

ここからは備忘録

もう疲弊しちゃいました.一旦アプリ開発しますわら。

ちなみにAWSは厳格にlinuxっぽいので、python3.6を作ってpackageをおとすOR aws-linuxつくって同じく落とす。ってのができます。

なぜか、できなかったので更に疲弊しちゃいました

もう、一回mongoでやります！

AWS

Posted by tanico.rikudo

奥鬼怒温泉旅行

AWS導入(1)