Advanced Logging of the Magento 2 Services

This entry was posted on October 24, 2019 by Vladimir Đurović, Backend Developer.

Magento 2 REST API

What is Web API?

According to Wikipedia, an API (Application Programming Interface) is a set of definitions of routines, protocols, and tools for building software and applications. Simply put, an API is a kind of interface that has a set of features that allow developers to access certain data of an application, operating system, or other services.

The Web API, as the name implies, is an API over the Internet that can be accessed by an HTTP. It’s a concept, not a technology. We can build a Web API using different technologies such as Java, .NET, etc. For example, Twitter's REST APIs provide a programmatic approach to reading and writing data that can integrate Twitter capabilities into our own application.

Web Service Types

The main protocol for transporting data between a web service and a client is HTTP, although other protocols can be used as well. The data format is XML or JSON.

SOAP

The first standard for publishing and consuming web services was the XML-based Simple Object Access Protocol (SOAP). Web clients would form HTTP requests and receive responses using the SOAP syntax. We will not deal with soap services in this article, but if you want to read more about them, you can click here.

RESTful API

The data is most commonly transferred in JSON format, although XML and YAML are also available. Based on the REST architecture, it is very flexible and easy to understand. It can be executed on any client or server that has HTTP / HTTPS support. RESTful services should have the following features:

  • Stateless
  • Cacheable
  • Uniform interface URI
  • Explicit use of HTTP methods
  • XML and / or JSON transfer

With this type of service, resources (e.g., static pages, files, databases) have their own URL or URI that identifies them. Access to resources is defined by the HTTP protocol, where each call performs one action (creates, reads, modifies, or deletes data). The same URL is used for all operations, but the HTTP method that defines the type of operation changes. REST uses CRUD-like HTTP methods such as GET, POST, PUT, DELETE, and OPTIONS.

Magento 2 API

The Magento Web API Framework provides integrators and developers with the means to use Web services that communicate with the Magento system. Magento supports both REST (representative state transfer) and SOAP (Simple Object Access Protocol). In Magento 2, the web API coverage is the same for REST and SOAP.

Magento 2 REST API

Is there some way to log all the requested REST APIs with details (requested URL, parameters, methods, timing, etc.) in Magento 2? It is a question that pesters many. The answer to this question is “There is no setting that can enable/disable logging API calls in Magento”, but you can do so by modifying a file that handles all the API calls and by logging the requests in a custom log file. 

This is useful when you know that there are many API calls being requested by the third-party applications to your website, but you are not sure what is actually being called, how many times, and at what time.

Let's get started. The first part is the creation of modules in the namespace SyncIt ApiRestLogger. The structure of the module folder should look like this:

 

Module Folder

 

In order for the Module to successfully register, it is necessary to contain the minimum of these three files: modul.xml, registration.php, and composer.json.

<?xml version="1.0" ?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="SyncIt_ApiRestLog" setup_version="1.0.0" schema_version="1.0.0" />
</config>

SyncIt/ApiRestLog/etc/modul.xml

 

<?php
use Magento\Framework\Component\ComponentRegistrar;

ComponentRegistrar::register(
    ComponentRegistrar::MODULE,
    'SyncIt_ApiRestLog',
    __DIR__
);

SyncIt/ApiRestLog/registration.php

 

{
    "name": "syncit/module-apirestlog",
    "description": "SyncIt Api Rest Logging Module",
    "type": "magento2-module",
    "authors": [
        {
            "email": "[email protected]",
            "name": "Vladimir Djurovic"
        }
    ],
    "minimum-stability": "dev",
    "require": {},
    "autoload": {
        "files": [
            "registration.php"
        ],
        "psr-4": {
            "SyncIt\\ApiRestLog\\": ""
        }
    }
}

SyncIt/ApiRestLog/composer.json

 

We will now hold onto the part that allows us to create logs. You do not want your API requests to log in *.log files that are not intended for this purpose. You need to create a Handler.php file in which you will specify the path where the log file is going to be placed.

<?php
namespace SyncIt\ApiRestLog\Model\Logger;

use Magento\Framework\Logger\Handler\Base;
use \Monolog\Logger;

/**
 * Class Handler
 * @package SyncIt\ApiRestLog\Model\Logger
 */
class Handler extends Base
{
    /**
     * Logging level
     * @var int
     */
    protected $loggerType = Logger::DEBUG;

    /**
     * File name
     * @var string
     */
    protected $fileName = '/var/log/syncit_rest_api.log';
}

SyncIt/ApiRestLog/Model/Logger/Handler.php

 

This will be a strange thing for many … an empty class. We use it as a virtual type in di.xml. If you want to know more about virtual types, you can check out the official Magento 2 documentation.

<?php
namespace SyncIt\ApiRestLog\Model\Logger;

/**
 * Class Logger
 * @package SyncIt\ApiRestLog\Model\Logger
 */
class Logger extends \Monolog\Logger
{
    #Code
}

SyncIt/ApiRestLog/Model/Logger/Logger.php

 

The system.xml file is to create the settings that can be set from the admin panel. In this case, you will have to create two fields <field>, one that will enable and disable the logic of your module and the other that tells you whether to log the request and response header or not.

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Config:etc/system_file.xsd">
    <system>
        <tab id="syncit" translate="label" sortOrder="2000" class="syncit-tab">
            <label>SyncIt</label>
        </tab>
        <section id="syncit_api_rest_logger" translate="label" type="text" sortOrder="10" showInDefault="1" showInStore="1" showInWebsite="1" >
            <label>API Rest Logger</label>
            <tab>syncit</tab>
            <class>syncit-settings</class>
            <resource>Magento_Backend::content</resource>
            <group id="general" translate="label" type="text" sortOrder="10" showInDefault="1" showInStore="1" showInWebsite="1" >
                <label>General</label>
                <field id="enabled" translate="label" type="select" sortOrder="11" showInDefault="1" showInStore="1" showInWebsite="1" >
                    <label>Enable</label>
                    <source_model>Magento\Config\Model\Config\Source\Yesno</source_model>
                </field>
                <field id="allowed_log_headers" translate="label" type="select" sortOrder="12" showInDefault="1" showInStore="1" showInWebsite="1" >
                    <label>Allowed Log Headers</label>
                    <comment/>
                    <source_model>Magento\Config\Model\Config\Source\Yesno</source_model>
                </field>
            </group>
        </section>
    </system>
</config>

SyncIt/ApiRestLog/etc/adminhtml/system.xml

 

In the following config.xml file, you define a default value for your settings. For example, the logic will be disabled after a module installation, while the header logging is allowed.

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Store:etc/config.xsd">
    <default>
        <syncit_api_rest_logger>
            <general>
                <enabled>0</enabled>
                <allowed_log_headers>1</allowed_log_headers>
            </general>
        </syncit_api_rest_logger>
    </default>
</config>

SyncIt/ApiRestLog/etc/config.xml

 

You have now reached the heart of your code. In this class, you should inject Logger through constructors of the class SyncIt\ApiRestLog\Plugin\RestApiLog. You can also safely use the RequestInterface methods because Magento is able to actually inject the class Magento\Framework\App\Request\Http when you use the following method:
Magento\Framework\HTTP\PhpEnvironment\Request. This object contains the information you need when logging in.

To begin with, you need to check if your module is enabled and if it is not you can cancel the further execution of the function. The example provided in the beforeDispatch function retrieves the following information: Store ID, Path, HTTP Method, Client Ip, Date, Requested Data, and of course header, if allowed. Finally, you need to convert the resulting data array to JSON and start writing the data onto a file.

    /**
    * @param Rest $subject
    * @param RequestInterface $request
    */
    public function beforeDispatch(Rest $subject, RequestInterface $request)
    {
        try {
            // If Enabled Api Rest Log
            if (!$this->_scopeConfig->getValue(self::API_LOGGER_ENABLED, ScopeInterface::SCOPE_STORE)) {
                return;
            }

            // Prepare Data For Log
            $requestedLogData = [
                'storeId' => $this->_storeManager->getStore()->getId(),
                'path' => $request->getPathInfo(),
                'httpMethod' => $request->getMethod(),
                'requestData' => $request->getContent(),
                'clientIp' => $request->getClientIp(),
                'date' => $this->_date->date()->format('Y-m-d H:i:s')
            ];

            // Log Headers
            if ($this->_scopeConfig->getValue(
self::API_LOGGER_ALLOWED_LOG_HEADERS, ScopeInterface::SCOPE_STORE)) {
                $requestedLogData['header'] = $this->getHeadersData(
$request->getHeaders());
            }

            // Logging Data
            $this->_logger->debug('Request = ' . $this->_serializer->serialize($requestedLogData));
        } catch (\Exception $exception) {
            $this->_logger->critical($exception->getMessage(), ['exception' => $exception]);
        }
    }

SyncIt/ApiRestLog/Plugin/RestApiLog.php (Part I)

 

Ok, the request is logged … but something seems to be missing. Of course - a response. In the same class, you can add one more function afterSendResponse, and it is called after sending a reply, as the name suggests. 

As in the example above, where you log the request, you are also logging the response with the data you want. In our example, we are logging status string, status code, and body data. Of course, if header logging is enabled, it will be logged in your log file.

    /**
    * @param Response $response
    * @param $result
    * @return mixed
    */
    public function afterSendResponse(Response $response, $result)
    {
        try {
            // If Enabled Api Rest Log
            if (!$this->_scopeConfig->getValue(self::API_LOGGER_ENABLED, ScopeInterface::SCOPE_STORE)) {
                return;
            }

            // Prepare Data For Log
            $requestedLogData = [
                'responseStatus' => $response->getReasonPhrase(),
                'responseStatusCode' => $response->getStatusCode(),
                'responseBody' => $response->getBody()
            ];

            // Log Headers
            if ($this->_scopeConfig->getValue(self::API_LOGGER_ALLOWED_LOG_HEADERS, ScopeInterface::SCOPE_STORE)) {
                $requestedLogData['header'] = $this->getHeadersData($response->getHeaders());
            }

            // Logging Data
            $this->_logger->debug('Response = ' . $this->_serializer->serialize($requestedLogData));
        } catch (\Exception $exception) {
            $this->_logger->critical($exception->getMessage(), ['exception' => $exception]);
        }
        return $result;
    }

SyncIt/ApiRestLog/Plugin/RestApiLog.php (Part II)

 

<?php
namespace SyncIt\ApiRestLog\Plugin;

use \Magento\Framework\App\Config\ScopeConfigInterface;
use \Magento\Framework\App\RequestInterface;
use \Magento\Framework\Stdlib\DateTime\TimezoneInterface;
use \Magento\Framework\Webapi\Rest\Response;
use \Magento\Framework\Serialize\SerializerInterface;
use \Magento\Store\Model\ScopeInterface;
use \Magento\Store\Model\StoreManagerInterface;
use \Magento\Webapi\Controller\Rest;
use SyncIt\ApiRestLog\Model\Logger\Logger;

/**
 * Class RestApiLog
 * @package SyncIt\ApiRestLog\Plugin
 */
class RestApiLog
{
    /**
     * Store Config Ids
     */
    const API_LOGGER_ENABLED = 'syncit_api_rest_logger/general/enabled';
    const API_LOGGER_ALLOWED_LOG_HEADERS = 'syncit_api_rest_logger/general/allowed_log_headers';

    /**
     * @var Logger
     */
    protected $_logger;

    /**
     * @var TimezoneInterface
     */
    protected $_date;

    /**
     * @var StoreManagerInterface
     */
    protected $_storeManager;

    /**
     * @var ScopeConfigInterface
     */
    protected $_scopeConfig;

    /**
     * @var SerializerInterface
     */
    private $_serializer;

    /**
     * RestApiLog constructor.
     * @param Logger $logger
     * @param TimezoneInterface $date
     * @param StoreManagerInterface $storeManager
     * @param ScopeConfigInterface $scopeConfig
     * @param SerializerInterface $serializer
     */
    public function __construct(
        Logger $logger,
        TimezoneInterface $date,
        StoreManagerInterface $storeManager,
        ScopeConfigInterface $scopeConfig,
        SerializerInterface $serializer
    )
    {
        $this->_logger = $logger;
        $this->_date = $date;
        $this->_storeManager = $storeManager;
        $this->_scopeConfig = $scopeConfig;
        $this->_serializer = $serializer;
    }

    /**
     * @param Rest $subject
     * @param RequestInterface $request
     */
    public function beforeDispatch(Rest $subject, RequestInterface $request)
    {
        try {
            // If Enabled Api Rest Log
            if (!$this->_scopeConfig->getValue(self::API_LOGGER_ENABLED, ScopeInterface::SCOPE_STORE)) {
                return;
            }

            // Prepare Data For Log
            $requestedLogData = [
                'storeId' => $this->_storeManager->getStore()->getId(),
                'path' => $request->getPathInfo(),
                'httpMethod' => $request->getMethod(),
                'requestData' => $request->getContent(),
                'clientIp' => $request->getClientIp(),
                'date' => $this->_date->date()->format('Y-m-d H:i:s')
            ];

            // Log Headers
            if ($this->_scopeConfig->getValue(self::API_LOGGER_ALLOWED_LOG_HEADERS, ScopeInterface::SCOPE_STORE)) {
                $requestedLogData['header'] = $this->getHeadersData($request->getHeaders());
            }

            // Logging Data
            $this->_logger->debug('Request = ' . $this->_serializer->serialize($requestedLogData));
        } catch (\Exception $exception) {
            $this->_logger->critical($exception->getMessage(), ['exception' => $exception]);
        }
    }

    /**
     * @param Response $response
     * @param $result
     * @return mixed
     */
    public function afterSendResponse(Response $response, $result)
    {
        try {
            // If Enabled Api Rest Log
            if (!$this->_scopeConfig->getValue(self::API_LOGGER_ENABLED, ScopeInterface::SCOPE_STORE)) {
                return;
            }

            // Prepare Data For Log
            $requestedLogData = [
                'responseStatus' => $response->getReasonPhrase(),
                'responseStatusCode' => $response->getStatusCode(),
                'responseBody' => $response->getBody()
            ];

            // Log Headers
            if ($this->_scopeConfig->getValue(self::API_LOGGER_ALLOWED_LOG_HEADERS, ScopeInterface::SCOPE_STORE)) {
                $requestedLogData['header'] = $this->getHeadersData($response->getHeaders());
            }

            // Logging Data
            $this->_logger->debug('Response = ' . $this->_serializer->serialize($requestedLogData));
        } catch (\Exception $exception) {
            $this->_logger->critical($exception->getMessage(), ['exception' => $exception]);
        }
        return $result;
    }

    /**
     * Method for getting all available data in header and convert them to array
     *
     * @param $headers
     * @return array
     */
    private function getHeadersData($headers): array
    {
        $headerLogData = [];
        foreach ($headers as $header) {
            $headerLogData[$header->getFieldName()] = $header->getFieldValue();
        }
        return $headerLogData;
    }
}

SyncIt/ApiRestLog/Plugin/RestApiLog.php (Full Class)

 

Your module is not finished yet. You must remember the empty class we have created before. Well, now you are actually going to use it. It is necessary to develop the di.xml file, and in it, to create the type of virtual class that was mentioned earlier.

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <type name="Magento\Webapi\Controller\Rest">
        <plugin name="rest-api-log" type="SyncIt\ApiRestLog\Plugin\RestApiLog"/>
    </type>
    <type name="Magento\Framework\Webapi\Rest\Response">
        <plugin name="rest-api-log" type="SyncIt\ApiRestLog\Plugin\RestApiLog" />
    </type>
    <type name="SyncIt\ApiRestLog\Model\Logger\Handler">
        <arguments>
            <argument name="filesystem" xsi:type="object">Magento\Framework\Filesystem\Driver\File</argument>
        </arguments>
    </type>
    <type name="SyncIt\ApiRestLog\Model\Logger\Logger">
        <arguments>
            <argument name="name" xsi:type="string">SyncIt_ApiRestLog</argument>
            <argument name="handlers"  xsi:type="array">
                <item name="system" xsi:type="object">SyncIt\ApiRestLog\Model\Logger\Handler</item>
            </argument>
        </arguments>
    </type>
</config>

SyncIt/ApiRestLog/etc/di.xml

 

Tag <plugin> tells Magento 2 to call the beforeDispatch method before calling the dispatch method. The dispatch method is the main entry point for REST API calls. If you want to know more about Prioritizing plugins you can take a look at Magento 2 dev docs.

Finally, you have come to the part where the magic is made. What is left is to enable the module and delete the cache. After these few commands, you can test the module. 

bin/magento module:enable SyncIt_ApiRestLog
bin/magento setup:upgrade
bin/magento cache:clean

After a couple of API calls, the log file should look something like this:

 

Store Info
1. Example of retrieving store information through the REST API. (Logged request and response)

 

That is how you can modify a file in order for it to handle all the API calls and log the requests in a custom log file. Should you need more help concerning the Magento 2 platform, you can contact us at [email protected] 

This entry was posted in Magento 2 and tagged Web Development, SyncIt Group, Magento 2, Web, Magento 2 Development, API, Magento 2 API, Magento 2 REST API, REST API, Advanced Logging, Magento Logging Services on October 24, 2019 by Vladimir Đurović, Backend Developer .