mirrors/poky
1
0
Fork 0
mirror of https://git.yoctoproject.org/poky synced 2026-07-16 03:47:03 +00:00
Code Issues Projects Releases Wiki Activity

bitbake: hashserv: Add gc-mark-stream command for batch hash marking

Browse Source 
Implements the `gc-mark-stream` command to allow for marking equivalence entries
in batch, by  making use of stream mode communication to the server.

The aim of this is to improve efficiency by reducing the impact of latency when
marking a high volume of hash entries.

Example usage of the new `gc-mark-stream` command:

```
$ cat << HASHES | \
./bin/bitbake-hashclient --address "ws://localhost:8688/ws" gc-mark-stream "alive"
unihash f37918cc02eb5a520b1aff86faacbc0a38124646
unihash af36b199320e611fbb16f1f277d3ee1d619ca58b
taskhash a1117c1f5a7c9ab2f5a39cc6fe5e6152169d09c0 method oe.sstatesig.OEOuthashBasic
HASHES
```

(Bitbake rev: c84715f28cd36666ea07a179d91b8c32ea0df8e7)

Signed-off-by: Alexander Marques <c137.marques@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Alexandre Marques
2025-03-12 11:22:04 +00:00
committed by Richard Purdie
parent cb9bff9eac
commit a4f8336982
4 changed files with 124 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bitbake/bin/bitbake-hashclient
+31
View File
@@ -227,6 +227,27 @@ def main():
print("New hashes marked: %d" % result["count"])
return 0
def handle_gc_mark_stream(args, client):
stdin = (l.strip() for l in sys.stdin)
marked_hashes = 0
try:
result = client.gc_mark_stream(args.mark, stdin)
marked_hashes = result["count"]
except ConnectionError:
logger.warning(
"Server doesn't seem to support `gc-mark-stream`. Sending "
"hashes sequentially using `gc-mark` API."
)
for line in stdin:
pairs = line.split()
condition = dict(zip(pairs[::2], pairs[1::2]))
result = client.gc_mark(args.mark, condition)
marked_hashes += result["count"]
print("New hashes marked: %d" % marked_hashes)
return 0
def handle_gc_sweep(args, client):
result = client.gc_sweep(args.mark)
print("Removed %d rows" % result["count"])
@@ -366,6 +387,16 @@ def main():
help="Keep entries in table where KEY == VALUE")
gc_mark_parser.set_defaults(func=handle_gc_mark)
gc_mark_parser_stream = subparsers.add_parser(
'gc-mark-stream',
help=(
"Mark multiple hashes to be retained for garbage collection. Input should be provided via stdin, "
"with each line formatted as key-value pairs separated by spaces, for example 'column1 foo column2 bar'."
)
)
gc_mark_parser_stream.add_argument("mark", help="Mark for this garbage collection operation")
gc_mark_parser_stream.set_defaults(func=handle_gc_mark_stream)
gc_sweep_parser = subparsers.add_parser('gc-sweep', help="Perform garbage collection and delete any entries that are not marked")
gc_sweep_parser.add_argument("mark", help="Mark for this garbage collection operation")
gc_sweep_parser.set_defaults(func=handle_gc_sweep)