opentelemetry全链路初探--日志跳转trace

前言

上一小节描述了metrics、traces，本小节来把log也加进去，并且做一个traces与log的联动

• 当查看日志的时候，可以同时跳转到对应的jaeger，查看分段trace情况

应用服务

• 本次要测试的应用服务架构为 a.py-->b.py
• 业务服务会往对应的目录打印日志，并且日志包含了trace_id

a.py

import tornado.httpserver as httpserver
import tornado.web
from tornado.ioloop import IOLoop
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.trace import get_tracer
from opentelemetry.propagate import inject
import requests
import loggingtrace.set_tracer_provider(TracerProvider(resource=Resource.create({SERVICE_NAME: "line-a"}))
)
span_processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://127.0.0.1:4318/v1/traces"))
trace.get_tracer_provider().add_span_processor(span_processor)logger = logging.getLogger('a.py')
logger.setLevel(logging.INFO)
formatter = logging.Formatter( '%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
file_handler = logging.FileHandler('logs/access-a.log', mode='a', encoding='utf-8')
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)def traced(name):def decorator(func):def wrapper(*args, **kwargs):tracer = get_tracer(__name__)with tracer.start_as_current_span(name):trace_id = format(trace.get_current_span().get_span_context().trace_id, '032x')logger.info('{} {}'.format(name, trace_id))return func(*args, **kwargs)return wrapperreturn decoratorclass TestFlow(tornado.web.RequestHandler):def get(self):views()self.finish('hello world')@traced("phase-1")
def views():views_2()headers = {}inject(headers)requests.get("http://127.0.0.1:20000", headers=headers)@traced("phase-2")
def views_2():passdef applications():urls = []urls.append([r'/', TestFlow])return tornado.web.Application(urls)def main():app = applications()server = httpserver.HTTPServer(app)server.bind(10000, '0.0.0.0')server.start(1)IOLoop.current().start()if __name__ == "__main__":try:main()except KeyboardInterrupt as e:IOLoop.current().stop()finally:IOLoop.current().close()

同理配置b.py

...
trace.set_tracer_provider(TracerProvider(resource=Resource.create({SERVICE_NAME: "line-b"}))
)logger = logging.getLogger('b.py')
logger.setLevel(logging.INFO)
formatter = logging.Formatter( '%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
file_handler = logging.FileHandler('logs/access-b.log', mode='a', encoding='utf-8')...
@traced("phase-3")
def views(headers):pass...
def main():app = applications()server = httpserver.HTTPServer(app)server.bind(20000, '0.0.0.0')server.start(1)IOLoop.current().start()
...

访问curl 127.0.0.1:10000之后，就会往日志文件中写入日志

▶ cat access-a.log
2025-08-21 11:22:18 - a.py - INFO - phase-1 38f0e8cef3e331993eca1675fa8c956b
2025-08-21 11:22:18 - a.py - INFO - phase-2 38f0e8cef3e331993eca1675fa8c956b▶ cat access-b.log
2025-08-21 11:22:18 - b.py - INFO - phase-3 38f0e8cef3e331993eca1675fa8c956b

日志系统loki

1）安装loki

loki.yaml

auth_enabled: falseserver:http_listen_port: 3100common:instance_addr: 127.0.0.1path_prefix: /lokistorage:filesystem:chunks_directory: /loki/chunksrules_directory: /loki/rulesreplication_factor: 1ring:kvstore:store: inmemoryschema_config:configs:- from: 2020-10-24store: tsdbobject_store: filesystemschema: v13index:prefix: index_period: 24hruler:alertmanager_url: http://localhost:9093

docker run -d --name grafana-loki \-p 3100:3100 \-v ./loki.yaml:/etc/loki/local-config.yaml \grafana/loki:v3.5.3

2）安装promtail

promtail.yaml

server:http_listen_port: 9080grpc_listen_port: 0positions:filename: /tmp/positions.yamlclients:- url: http://10.22.12.178:3100/loki/api/v1/pushscrape_configs:
- job_name: systemstatic_configs:- labels:job: access_log__path__: /var/log/access*.log

docker run -d --name grafana-promtail \-v ./promtail.yaml:/etc/promtail/config.yml \-v ./logs:/var/log:ro \grafana/promtail:3.5

3）安装grafana

docker run -d --name grafana-grafana \-p 3000:3000 \grafana/grafana:12.1.1

4）在grafana上配置loki

登录grafana --> connections --> data sources --> 选择loki --> 开始配置

最关键的步骤就是把connection改成loki的地址10.22.12.178:3100

5）查看下刚才写入的日志

日志已经进来了

logs跳转至traces

之前已经将traces采集进jaeger，现在又拥有了日志，将他们通过traceid联动起来，在grafana找到日志，并且通过traceid字段跳转去jaeger查询详细信息

登录grafana --> connections --> data sources --> 选择刚才配置好的loki --> Derived fields

• Name：随便取一个
• Type：Regex in log line
• Regex：([0-9a-f]{32,})
• URL：配置jaeger的URL，http://10.22.12.178:16686/trace/${__value.raw}

配置完成之后查看日志，有会个跳转按钮，点击之后就来带jaeger-UI的查询页面

联系我

• 联系我，做深入的交流
  

---

至此，本文结束
在下才疏学浅，有撒汤漏水的，请各位不吝赐教...