log4j2⾃动删除过期⽇志⽂件配置及实现原理解析
  ⽇志⽂件⾃动删除功能必不可少，当然你可以让运维去做这事，只是这不地道。⽽⽇志组件是⼀个必备组件，让其多做⼀件删除的⼯作，⽆可厚⾮。本⽂就来探讨下 log4j 的⽇志⽂件⾃动删除实现吧。
0. ⾃动删除配置参考样例: （l）
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration status="warn" monitorInterval="30" strict="true"
schema="Log4J-V2.2.xsd">
<Properties>
<Property name="log_level">info</Property>
</Properties>
<Appenders>
<!-- 输出到控制台 -->
<Console name="Console" target="SYSTEM_OUT">
<ThresholdFilter level="${log_level}" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%t] %p - %c - %m%n"/>
</Console>
<!-- 与properties⽂件中位置存在冲突，如有问题，请注意调整 -->
<RollingFile name="logFile" fileName="logs/app/test.log"
filePattern="logs/app/history/test-%d{MM-dd-yyyy}-%">
<ThresholdFilter level="${log_level}" onMatch="ACCEPT" onMismatch="DENY"/>
<PatternLayout pattern="%d{yyyy-MM-dd'T'HH:mm:ss.SSS} [%p] [%c:%L] -- %m%n"/>
<Policies>
<!-- 按天递计算频率 -->
<TimeBasedTriggeringPolicy interval="1"/>
<SizeBasedTriggeringPolicy size="500 MB"/>
<OnStartupTriggeringPolicy />
</Policies>
<!-- 删除策略配置 -->
<DefaultRolloverStrategy max="5">
<Delete basePath="logs/app/history" maxDepth="1">
<IfFileName glob="*."/>
<IfLastModified age="7d"/>
</Delete>
<Delete basePath="logs/app/history" maxDepth="1">
<IfFileName glob="*.docx"/>
</Delete>
<Delete basePath="logs/app/history" maxDepth="1">
<IfFileName glob="*.vsdx"/>
</Delete>
</DefaultRolloverStrategy>
</RollingFile>
<Async name="Async" bufferSize="2000" blocking="false">
<AppenderRef ref="logFile"/>
</Async>
</Appenders>
<Loggers>
<Root level="${log_level}">
<AppenderRef ref="Console"/>
<AppenderRef ref="Async"/>
</Root>
<!-- 配置个例 -->
<Logger name="filter" level="info"/>
</Loggers>
</Configuration>
  如果仅想停留在使⽤层⾯，如上l配置⽂件⾜矣！
  不过，⾄少得注意⼀点，以上配置需要基于log4j2, ⽽如果你是 log4j1.x，则需要做下⽆缝升级：主要就是换下jar包版本，换个桥接包之类的,⽐如下参考配置：
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.2</version>
</dependency>
<!-- 桥接：告诉commons logging使⽤Log4j2 -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>1.7.26</version>
</dependency>
<!-- 此处⽼版本，需注释掉 -->
<!--<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>-->
<dependency>
<groupId>org.apachemons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.10</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-web</artifactId>
<version>2.8.2</version>
</dependency>
  如果还想多了解⼀点其运⾏原理，就跟随本⽂的脚步吧:
1. ⾃动清理⼤体运⾏流程
  ⾃动删除⼯作的运⾏原理⼤体流程如下。（⼤抵都是如此）
1. 加载log4j
    2. 读取appenders，并添加到log4j上下⽂中;
    3. 加载 policy, 加载 rollover 配置;
    4. 写⼊⽇志时判断是否满⾜rollover配置, 默认是⼀天运⾏⼀次, 可⾃⾏添加各种运⾏测试, ⽐如⼤⼩、启动时;  所以，删除策略的核⼼是每⼀次添加⽇志时。代码验证如下：
// 在每次添加⽇志时判定log4j2 配置详解properties
// org.apache.appender.RollingRandomAccessFileAppender#append
/**
* Write the log entry rolling over the file when required.
*
* @param event The LogEvent.
*/
@Override
public void append(final LogEvent event) {
final RollingRandomAccessFileManager manager = getManager();
// 重点：直接检查是否需要 rollover, 如需要直接进⾏
manager.checkRollover(event);
// Leverage the nice batching behaviour of async Loggers/Appenders:
// we can signal the file manager that it needs to flush the buffer
// to disk at the end of a batch.
// From a user's point of view, this means that all log events are
/
/ _always_ available in the log file, without incurring the overhead
// of immediateFlush=true.
manager.setEndOfBatch(event.isEndOfBatch()); // FIXME manager's EndOfBatch threadlocal can be deleted
// LOG4J2-1292 utilize de() method: taken care of in superclass
super.append(event);
}
// org.apache.lling.RollingFileManager#checkRollover
/**
* Determines if a rollover should occur.
* @param event The LogEvent.
*/
public synchronized void checkRollover(final LogEvent event) {
// 由各触发策略判定是否需要进⾏ rolling
// 如需要, 则调⽤ rollover()
if (triggeringPolicy.isTriggeringEvent(event)) {
rollover();
}
}
  所以，何时进⾏删除？答案是在适当的时机，这个时机可以是任意时候。
2. log4j ⽇志滚动
  ⽇志滚动，可以是重命名，也可以是删除⽂件。但总体判断是否可触发滚动的前提是⼀致的。我们这⾥主要关注⽂件删除。我们以时间作为依据看下判断过程。
// 1. 判断是否是触发事件时机
// org.apache.lling.TimeBasedTriggeringPolicy#isTriggeringEvent
/**
* Determines whether a rollover should occur.
* @param event  A reference to the currently event.
* @return true if a rollover should occur.
*/
@Override
public boolean isTriggeringEvent(final LogEvent event) {
if (FileSize() == 0) {
return false;
}
final long nowMillis = TimeMillis();
// TimeBasedTriggeringPolicy, 是基于时间判断的, 此处为每天⼀次
if (nowMillis >= nextRolloverMillis) {
nextRolloverMillis = PatternProcessor().getNextTime(nowMillis, interval, modulate);
return true;
}
return false;
}
// org.apache.lling.RollingFileManager#rollover()
public synchronized void rollover() {
if (!hasOutputStream()) {
return;
}
// strategy 是xml配置的策略
if (rollover(rolloverStrategy)) {
try {
size = 0;
initialTime = System.currentTimeMillis();
createFileAfterRollover();
} catch (final IOException e) {
logError("Failed to create file after rollover", e);
}
}
}
// RollingFileManager 统⼀管理触发器
// org.apache.lling.RollingFileManager#rollover
private boolean rollover(final RolloverStrategy strategy) {
boolean releaseRequired = false;
try {
// Block until the asynchronous operation is completed.
// 上锁保证线程安全
semaphore.acquire();
releaseRequired = true;
} catch (final InterruptedException e) {
logError("Thread interrupted while attempting to check rollover", e);
return false;
}
boolean success = true;
try {
// 由各触发器运⾏ rollover 逻辑
final RolloverDescription descriptor = llover(this);
if (descriptor != null) {
writeFooter();
closeOutputStream();
if (Synchronous() != null) {
LOGGER.debug("RollingFileManager executing synchronous {}", Synchronous());
try {
// 先使⽤同步⽅法，改名，然后再使⽤异步⽅法操作更多
success = Synchronous().execute();
} catch (final Exception ex) {
success = false;
logError("Caught error in synchronous task", ex);
}
}
// 如果配置了异步器, 则使⽤异步进⾏ rollover
if (success && Asynchronous() != null) {
LOGGER.debug("RollingFileManager executing async {}", Asynchronous());
// CompositeAction, 使⽤异步线程池运⾏⽤户的 action
// 在异步运⾏action期间，锁是不会被释放的，以避免线程安全问题
// 直到异步任务完成，再主动释放锁
releaseRequired = false;
}
return true;
}
return false;
} finally {
if (releaseRequired) {
}
}
}
  此处滚动有两个处理点，1. 每个滚动策略可以⾃⾏处理业务； 2. RollingFileManager 统⼀管理触发同步和异步的滚动action；
3. DefaultRolloverStrategy 默认滚动策略驱动
  DefaultRolloverStrategy 作为⼀个默认的滚动策略实现，可以配置多个 Action, 然后处理删除操作。
  删除有两种⽅式: 1. 当次滚动的⽂件数过多，会⽴即进⾏删除; 2. 配置单独的 DeleteAction, 根据配置的具体策略进⾏删除。（但该Action只会被返回给外部调⽤，⾃⾝则不会执⾏）
// org.apache.lling.DefaultRolloverStrategy#rollover
/**
* Performs the rollover.
*
* @param manager The RollingFileManager name for current active log file.
* @return A RolloverDescription.
* @throws SecurityException if an error occurs.
*/
@Override
public RolloverDescription rollover(final RollingFileManager manager) throws SecurityException {
int fileIndex;
/
/ 默认 minIndex=1
if (minIndex == Integer.MIN_VALUE) {
final SortedMap<Integer, Path> eligibleFiles = getEligibleFiles(manager);
fileIndex = eligibleFiles.size() > 0 ? eligibleFiles.lastKey() + 1 : 1;
} else {
if (maxIndex < 0) {
return null;
}
final long startNanos = System.nanoTime();
// 删除case1: 获取符合条件的⽂件数，同时清理掉⼤于  max 配置的⽇志⽂件
// 如配置 max=5, 当前只有4个满⾜时, 不会⽴即清理⽂件, 但也不会阻塞后续流程
/
/ 只要没有出现错误, fileIndex 不会⼩于0
fileIndex = purge(minIndex, maxIndex, manager);
if (fileIndex < 0) {
return null;
}
if (LOGGER.isTraceEnabled()) {
final double durationMillis = Millis(System.nanoTime() - startNanos);
}
}
// 进⼊此区域即意味着，必然有⽂件需要滚动，重新命名了
final StringBuilder buf = new StringBuilder(255);
final String currentFileName = FileName();
String renameTo = String();
final String compressedName = renameTo;
Action compressAction = null;
FileExtension fileExtension = FileExtension();
if (fileExtension != null) {
renameTo = renameTo.substring(0, renameTo.length() - fileExtension.length());
compressAction = ateCompressAction(renameTo, compressedName,
true, compressionLevel);
}
// 未发⽣⽂件重命名情况，即⽂件未被重命名未被滚动
// 该种情况应该不太会发⽣
if (currentFileName.equals(renameTo)) {
LOGGER.warn("Attempt to rename file {} to itself will be ignored", currentFileName);
return new RolloverDescriptionImpl(currentFileName, false, null, null);
}
// 新建⼀个重命令的 action, 返回待⽤
final FileRenameAction renameAction = new FileRenameAction(new File(currentFileName), new File(renameTo),
manager.isRenameEmptyFiles());
// 异步处理器，会处理⽤户配置的异步action，如本⽂配置的 DeleteAction
// 它将会在稍后被提交到异步线程池中运⾏
final Action asyncAction = merge(compressAction, customActions, stopCustomActionsOnError);
// 封装Rollover返回, renameAction 是同步⽅法, 其他⽤户配置的动态action 则是异步⽅法
// 删除case2: 封装异步返回action
return new RolloverDescriptionImpl(currentFileName, false, renameAction, asyncAction);
}
private int purge(final int lowIndex, final int highIndex, final RollingFileManager manager) {
// 默认使⽤ accending 的⽅式进⾏清理⽂件
return useMax ? purgeAscending(lowIndex, highIndex, manager) : purgeDescending(lowIndex, highIndex, manager);
}
// org.apache.lling.DefaultRolloverStrategy#purgeAscending
/**
* Purges and renames old log files in preparation for rollover. The oldest file will have the smallest index, the
* newest the highest.
*
* @param lowIndex low index. Log file associated with low index will be deleted if needed.
* @param highIndex high index.
* @param manager The RollingFileManager
* @return true if purge was successful and rollover should be attempted.
*/
private int purgeAscending(final int lowIndex, final int highIndex, final RollingFileManager manager) {
final SortedMap<Integer, Path> eligibleFiles = getEligibleFiles(manager);
final int maxFiles = highIndex - lowIndex + 1;
boolean renameFiles = false;
// 依次迭代 eligibleFiles, 删除
while (eligibleFiles.size() >= maxFiles) {
try {
LOGGER.debug("Eligible files: {}", eligibleFiles);
Integer key = eligibleFiles.firstKey();
LOGGER.debug("Deleting {}", (key).toFile().getAbsolutePath());
// 调⽤nio的接⼝删除⽂件
Files.(key));
renameFiles = true;
} catch (IOException ioe) {
<("Unable to delete {}, {}", eligibleFiles.firstKey(), Message(), ioe);
break;
}
}
final StringBuilder buf = new StringBuilder();
if (renameFiles) {
// 针对未完成删除的⽂件，继续处理
// ⽐如使⽤匹配的⽅式匹配⽂件, 则不能被正常删除
// 还有些未超过maxFiles的⽂件
for (Map.Entry<Integer, Path> entry : Set()) {
buf.setLength(0);
// LOG4J2-531: directory scan & rollover must use same format
String currentName = Value().toFile().getName();
String renameTo = String();
int suffixLength = suffixLength(renameTo);
if (suffixLength > 0 && suffixLength(currentName) == 0) {
renameTo = renameTo.substring(0, renameTo.length() - suffixLength);
}
Action action = new Value().toFile(), new File(renameTo), true);
try {
LOGGER.debug("DefaultRolloverStrategy.purgeAscending executing {}", action);
if (!ute()) {
return -1;
}
} catch (final Exception ex) {
LOGGER.warn("Exception during purge in RollingFileAppender", ex);
return -1;
}
}
}
/
/ 此处返回的 findIndex ⼀定是 >=0 的
return eligibleFiles.size() > 0 ?
(eligibleFiles.lastKey() < highIndex ? eligibleFiles.lastKey() + 1 : highIndex) : lowIndex;
}
4. 符合过滤条件的⽂件查
  当配置了 max 参数，这个参数是如何匹配的呢？⽐如我某个⽂件夹下有很历史⽂件，是否都会匹配呢？// ⽂件查规则
// org.apache.lling.AbstractRolloverStrategy#getEligibleFiles
protected SortedMap<Integer, Path> getEligibleFiles(final RollingFileManager manager) {
return getEligibleFiles(manager, true);
}

版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。