July 24, 2024

hackergotchi for Kentaro Hayashi

Kentaro Hayashi

apt-upgrade-canary - PoC apt JSON hook use case

apt-upgrade-canary is a helper program to alert when upgrading packages via apt.

If there are some packages which causes a critical or serious bug, it shows warnings for terminal.

Then you can stay on current version of package by canceling upgrades.

This program is aimed to help not to installing problematic packages via manual upgrades.

Usually, you should delegate package upgrade to unattended-upgrades and apt-listbugs.

apt-listbugs kindly warns if there are known bugs.

In most cases it is enough safer.

But there is one exception, it is not true that always apt-listbugs can follow very latest critical or serious bugs in timely manner.

apt-upgrade-canary checks UDD mirror database (it take into account to cache query not to throw redundant one frequently)

Getting started

  1. Install required packages.
$ sudo apt install -y make ruby-pg ruby-json ruby-term-ansicolor
  1. Clone https://salsa.debian.org/kenhys/apt-upgrade-canary.

  2. Just sudo make install.

Use case in action

If there is no serious bugs which is reported against target packages, apt-upgrade-canary reports no error.

apt-upgrade-canary: no problem

If there are something weird happen, it reports matched bugs.

apt-upgrade-canary: serious case

So you can stay on.

24 July, 2024 12:03PM

hackergotchi for Gunnar Wolf

Gunnar Wolf

DebConf24 Continuous Key-Signing Party

🎉🥳🤡🎂🍥 Yay, party! 🎉🥳🤡🎂🍥

🎉🥳🤡🎂🍥 Yay, crypto! 🎉🥳🤡🎂🍥

DebCamp has started, and in a couple of days, we will fully be in DebConf24 mode!

As most of you know, an important part that binds Debian together is our cryptographic identity assurance, and that is in good measure tightened by the Continuous Key-Signing Parties we hold at DebConfs and other Debian and Free Software gatherings.

As I have done during (most of) the past DebConfs, I have prepared a set of pseudo-social maps to help you find where you are in the OpenPGP mesh of our conference. Naturally, Web-of-Trust maps should be user-centered, so find your own at:

https://people.debian.org/~gwolf/dc24_ksp/

The list is now final and it will not receive any modifications (I asked for them some days ago); if your name still appears on the list and you don’t want to be linked to the DC24 KSP in the future, tell me and I’ll remove it from future versions of the list (but it is part of the final DC24 file, as its checksum is already final)

Speaking of which!

If you are to be a part of the keysigning, get the final DC24 file and, on a device you trust, check its SHA256 by running:

$ sha256sum dc24_fprs.txt
11daadc0e435cb32f734307b091905d4236cdf82e3b84f43cde80ef1816370a5  dc24_fprs.txt

Make sure the resulting number matches the one I’m presenting. If it doesn’t, ensure your copy of the file is not corrupted (i.e. download again). If it still doess not match, notify me immediately.

Does any of the above confuse you? Please come to (or at least, follow the stream for) my session on DebConf opening day, Continuous Key-Signing Party introduction, 10:30 Korean time; I will do my best to explain the details to you.

PS- I will soon provide a simple, short PDF that will probably be mass-printed at FrontDesk so that you can easily track your KSP progress.

24 July, 2024 03:25AM

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RQuantLib 0.4.23 on CRAN: Updates

A new minor release 0.4.23 of RQuantLib just arrived at CRAN earlier today, and will be uploaded to Debian in due course.

QuantLib is a rather comprehensice free/open-source library for quantitative finance. RQuantLib connects (some parts of) it to the R environment and language, and has been part of CRAN for more than twenty-two years (!!) as it was one of the first packages I uploaded.

This release of RQuantLib updates to QuantLib version 1.35 released this morning. It accommodates some removals following earlier deprecations, and also updates most of the code in the function for a more readable and compact form of creating shared pointers via make_shared() along with auto.

Changes in RQuantLib version 0.4.23 (2024-07-23)

  • Adjustments for QuantLib 1.35 and removal of deprecated code (in utility functions and dividend case of vanilla options)

  • Adjustments for new changes in QuantLib 1.35

  • Refactoring most C++ files making more use of both auto and make_shared to simplify and shorten expressions

Courtesy of my CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 July, 2024 02:48AM

qlcal 0.0.12 on CRAN: Calendar Updates

The twelveth release of the qlcal package arrivied at CRAN today.

qlcal delivers the calendaring parts of QuantLib. It is provided (for the R package) as a set of included files, so the package is self-contained and does not depend on an external QuantLib library (which can be demanding to build). qlcal covers over sixty country / market calendars and can compute holiday lists, its complement (i.e. business day lists) and much more. Examples are in the README at the repository, the package page, and course at the CRAN package page.

This releases synchronizes qlcal with the QuantLib release 1.35 (made today) and contains more updates to 2024 calendars.

Changes in version 0.0.12 (2024-07-22)

  • Synchronized with QuantLib 1.35 released today

  • Calendar updates for Chile, India, United States, Brazil

Courtesy of my CRANberries, there is a diffstat report for this release. See the project page and package documentation for more details, and more examples. If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 July, 2024 12:45AM

July 23, 2024

Russell Coker

More About Kogan 5120*2160 Monitor

On the 18th of May I blogged about my new 5120*2160 monitor [1]. One thing I noted was that one Netflix movie had run in an aspect ratio that used all space on the monitor. I still don’t know if the movie in question was cropped in a letterbox manner but other Netflix shows in “full screen” mode don’t extend to both edges. Also one movie I downloaded as in 3840*1608 resolution which is almost exactly the same aspect ratio as my monitor. I wonder if some company is using 5120*2160 screens for TVs, 4K and FullHD are rumoured to be cheaper than most other resolutions partly due to TV technology being used for monitors. There is the Anamorphic Format of between 2.35:1 and 2.40:1 [2] which is a close match for the 2.37:1 of my new monitor.

I tried out the HDMI audio on a Dell laptop and my Thinkpad Yoga Gen3 and found it to be of poor quality, it seemed limited to 2560*1440, at this time I’m not sure how much of the fault is due to the laptops and how much is due to the monitor. The monitor docs state that it needs HDMI version 2.1 which was released in 2017 and my Thinkpad Yoga Gen3 was released in 2018 so probably doesn’t have that. The HDMI cable in question did 4K resolution on my previous monitor so it should all work at a minimum of 4K resolution.

The switching between inputs is a problem. If I switch between DisplayPort for my PC and HDMI for a laptop the monitor will usually timeout before the laptop establishes a connection and then switch back to the DisplayPort input. So far I have had to physically disconnect the input source I don’t want to use. The DisplayPort switch that I’ve used doesn’t seem designed to work with resolutions higher than 4K.

I’ve bought a new USB-C dock which is described as doing 8K which means that as my Thinkpad is described as supporting 5120×2880@60Hz over USB-C I should be able to get 5120*2160 without any problems, however for unknown reasons I only get 4K. For work I’m using a Dell Latitude 7400 2in1 that’s apparently only capable of 4096*2304 @24 Hz which is less pixels than 5120*2160 and it will also only do 4K resolution. But for both those cases it’s still a significant improvement over 2560*1440. I tested with a Dell Latitude 7440 which gave the full 5120*2160 resolution, I was unable to find specs on what the maximum resolution of the 7440 is. I also have bought DisplayPort switch rated at 8K resolution. I got a switch that doesn’t also do USB because the ones that do 8K resolution and USB are about $70. The only KVM switch I saw for 8K resolution at a reasonable price was one designed for switching between two laptops and there doesn’t seem to be any adaptors to convert from regular DisplayPort to USB-C alternative mode so that wasn’t viable. Currently I have the old KVM switch used for USB only (for keyboard and mouse) and the new switch which only does DisplayPort. So I have two buttons to push when switching between input sources which isn’t too bad.

It seems that for PCs resolutions with more pixels than 4K are as difficult and inconvenient now as 4K was 6 years ago when I started doing it. If you want higher than 4K resolution to just work at this time then you need Apple hardware.

The monitor has a series of modes for different types of output, I’ve found “standard” to be good for text and “movie” to be good for watching movies/TV and for playing RTS games. I previously wrote about how to use ddcutil to use a monitor as a KVM switch [3], unfortunately I can’t do this with the new monitor as the time that the monitor waits for a good signal on a new input after changing is shorter than the time taken for Linux on the laptops I’m using to enable HDMI output. I’ve found the following commands to do the basics.

# get display mode
ddcutil getvcp DC
# set standard mode
ddcutil setvcp DC 0
# set movie mode
ddcutil setvcp DC 03

Now that I have that going the next thing I want to do is to have it switch between “standard” and “movie” modes when I switch keyboard focus.

23 July, 2024 09:34AM by etbe

July 22, 2024

Blog Comments

The Akismet WordPress anti-spam plugin has changed it’s policy to not run on sites that have adverts which includes mine. Without it I get an average of about 1 spam comment per hour and the interface for removing spam takes more mouse actions than desired. For email spam it’s about the same volume half of which is messages with SpamAssassin scores high enough to go into the MaybeSpam folder (that I go through every few weeks) and half of which goes straight to my inbox. But fortunately moving spam to a folder where I can later use it to train Bayesian classification is a much faster option on PC and is also something I can do from my phone MUA.

As an experiment I have configured my blog to only take comments from registered users. It will be interesting to see how many spammers make it through that and to also see feedback from genuine people. People who can’t comment can tell me about it via the contact methods listed here [1].

I previously wrote about other ways of dealing with hostile comments [2]. Blogging seems to be less popular nowadays so a Planet specific forum doesn’t seem a viable option. It’s a pity, I think that YouTube and Facebook have taken over from blogs and that’s not a good thing.

22 July, 2024 10:05PM by etbe

hackergotchi for Martin-Éric Racine

Martin-Éric Racine

dhcpcd replacing dhclient for Trixie... or perhaps networkd?

My work on overhauling dhcpcd as the prime replacement for ISC's discontinued DHCP client is done. The package has achieved stability, both upstream and at Debian. The only remaining points are bug #1038882 to swap the Priorities of isc-dhcp-client and dhcpcd-base in the repository's override, and swaping ifupdown's search order to put dhcpcd first.

Meanwhile, ifupdown's de-facto maintainer prompted me to sollicit opinions on which of the 4 ifupdown implementations should ship with a minimal installation for Trixie. This, in turn, re-opened the debate of what should be Debian's default network configuation framework (see the thread starting with this post).

networkd

Given how most Debian ports (except for Hurd) ship with systemd, which includes a disabled networkd by standard, many people in the thread feel that this should become the default network configuration tool for minimal installations. As it happens, most of my hosts fit that scenario, so I figured that I would give another go at testing networkd on one host.

I used the following minimalistic /etc/systemd/network/dhcp.network:

[Match]
Name=en* wl*

[Network]
DHCP=yes

This correctly configured IPv4 via DHCP, with the small caveat that it doesn't update /etc/resolv.conf without installing resolvconf or systemd-resolved.

However, networkd's default IPv6 settings really are not suitable for public consumption. The key issues (see Bug #1076432):

  1. Temporary addresses are not enabled by default. Worse, the setting is ignored if it was enabled by sysctl during bootup. This is a major privacy issue. Adding IPv6PrivacyExtensions=yes to the above exposed another issue: instead of using the fe80 address generated by the kernel, networkd adds a new one.
  2. Networkd uses EUI64 addresses by default. This is another major privacy issue, since EUI64 addresses are forensically traceable to the interface's MAC address. Worse, the setting is ignored if stable-privacy was enabled by sysctl during bootup. To top it all, networkd does stable-privacy using systemd's time-proven brain-dead approach of reinventing the wheel: instead of merely setting the kernel's address generation mode to 3 and letting it configure the secret address, it expects the secret address to be spelled out in the systemd unit.

Conclusion: networkd works well enough for someone configuring an IPv4-only network from 20 years ago, but is utterly inadequate for IPv6 or dual-stack installations, doubly so on a software distribution that claims to care about privacy and network security.

22 July, 2024 06:38PM by Martin-Éric (noreply@blogger.com)

July 21, 2024

hackergotchi for Mike Gabriel

Mike Gabriel

Polis - a FLOSS Tool for Civic Participation -- Issues extending Polis and adjusting our Goals

Here comes the 3rd article of the 5-episode blog post series on Polis, written by Guido Berhörster, member of staff at my company Fre(i)e Software GmbH.

Enjoy also this read on Guido's work on Polis,
Mike

Table of Contents of the Blog Post Series

  1. Introduction
  2. Initial evaluation and adaptation
  3. Issues extending Polis and adjusting our goals (this article)
  4. Creating (a) new frontend(s) for Polis
  5. Current status and roadmap

Polis - Issues extending Polis and adjusting our Goals

After the initial implementation of limited branding support, user feedback and the involvement of an UX designer lead to the conclusion that we needed more far-reaching changes to the user interface in order to reduce visual clutter, rearrange and improve UI elements, and provide better integration with the websites in which conversations are embedded.

Challenges when visualizing Data in Polis

Polis visualizes groups using a spatial projection of users based on similarities in voting behavior and places them in two to five groups using a clustering algorithm. During our testing and evaluation users were rarely able to interpret the visualization and often intuitively made incorrect assumptions e.g. by associating the filled area of a group with its significance or size. After consultation with a member of the Multi-Agent Systems (MAS) Group at the University of Groningen we chose to temporarily replace the visualization offered by Polis with simple bar charts representing agreement or disagreement with statements of a group or the majority. We intend to revisit this and explore different forms of visualization at a later point in time.

The different factors playing into the weight attached to statements which determine the pseuodo-random order in which they are presented for voting (“comment routing”) proved difficult to explain to stakeholders and users and the admission of the ad-hoc and heuristic nature of the used algorithm1 by Polis’ authors lead to the decision to temporarily remove this feature. Instead, statements should be placed into three groups, namely

  1. metadata questions,
  2. seed statements,
  3. and participant statements

Statements should then be sorted by group but in a fully randomized order within the group so that metadata questions would be presented before seed statements which would be presented before participant’s statements. This simpler method was deemed sufficient for the scale of our pilot projects, however we intend to revisit this decision and explore different methods of “comment routing” in cooperation with our scientific partners at a later point in time.

An evaluation of the requirements for implementing mandatory authentication and adding support for additional authentication methods to Polis showed that significant changes to both the administration and participation frontend were needed due to a lack of an abstraction layer or extension mechanism and the current authentication providers being hardcoded in many parts of the code base.

A New Frontend is born: Particiapp

Based on the implementation details of the participation frontend, the invasive nature of the changes required, and the overhead of keeping up with active upstream development it became clear that a different, more flexible approach to development was needed. This ultimately lead to the creation of Particiapp, a new Open Source project providing the building blocks and necessary abstraction layers for rapid protoyping and experimentation with different fontends which are compatible with but independent from Polis.

  1. Small, Christopher T., Bjorkegren, Michael, Erkkilä, Timo, Shaw, Lynette and Megill, Colin (2021). Polis: Scaling deliberation by mapping high dimensional opinion spaces. Recerca. Revista de Pensament i Anàlisi, 26(2), pp. 1-26. 

21 July, 2024 12:58PM by sunweaver

Russell Coker

SE Linux Policy for Dell Management

The recent issue of Windows security software killing computers has reminded me about the issue of management software for Dell systems. I wrote policy for the Dell management programs that extract information from iDRAC and store it in Linux. After the break I’ve pasted in the policy. It probably needs some changes for recent software, it was last tested on a PowerEdge T320 and prior to that was used on a PowerEdge R710 both of which are old hardware and use different management software to the recent hardware. One would hope that the recent software would be much better but usually such hope is in vain. I deliberately haven’t submitted this for inclusion in the reference policy because it’s for proprietary software and also it permits many operations that we would prefer not to permit.

The policy is after the break because it’s larger than you want on a Planet feed. But first I’ll give a few selected lines that are bad in a noteworthy way:

  1. sys_admin means the ability to break everything
  2. dac_override means break Unix permissions
  3. mknod means a daemon creates devices due to a lack of udev configuration
  4. sys_rawio means someone didn’t feel like writing a device driver, maintaining a device driver for DKMS is hard and getting a driver accepted upstream requires writing quality code, in any case this is a bad sign.
  5. self:lockdown is being phased out, but used to mean bypassing some integrity protections, that would usually be related to sys_rawio or similar.
  6. dev_rx_raw_memory is bad, reading raw memory allows access to pretty much everything and execute of raw memory is something I can’t imagine a good use for, the Reference Policy doesn’t use this anywhere!
  7. dev_rw_generic_chr_files usually means a lack of udev configuration as udev should do that.
  8. storage_raw_write_fixed_disk shouldn’t be needed for this sort of thing, it doesn’t do anything that involves managing partitions.

Now without network access or other obvious ways of remote control this level of access while excessive isn’t necessarily going to allow bad things to happen due to outside attack. But if there are bugs in the software there’s nothing to stop it from giving the worst results.

allow dell_datamgrd_t self:capability { dac_override dac_read_search mknod sys_rawio sys_admin };
allow dell_datamgrd_t self:lockdown integrity;
dev_rx_raw_memory(dell_datamgrd_t)
dev_rw_generic_chr_files(dell_datamgrd_t)
dev_rw_ipmi_dev(dell_datamgrd_t)
dev_rw_sysfs(dell_datamgrd_t)
storage_raw_read_fixed_disk(dell_datamgrd_t)
storage_raw_write_fixed_disk(dell_datamgrd_t)

allow dellsrvadmin_t self:lockdown integrity;
allow dellsrvadmin_t self:capability { sys_admin sys_rawio };
dev_read_raw_memory(dellsrvadmin_t)
dev_rw_sysfs(dellsrvadmin_t)
dev_rx_raw_memory(dellsrvadmin_t)

The best thing that Dell could do for their customers is to make this free software and allow the community to fix some of these issues.


Here is dellsrvadmin.te:

policy_module(dellsrvadmin,1.0.0)

require {
  type dmidecode_exec_t;
  type udev_t;
  type device_t;
  type event_device_t;
  type mon_local_test_t;
}

type dellsrvadmin_t;
type dellsrvadmin_exec_t;
init_daemon_domain(dellsrvadmin_t, dellsrvadmin_exec_t)

type dell_datamgrd_t;
type dell_datamgrd_exec_t;
init_daemon_domain(dell_datamgrd_t, dell_datamgrd_t)

type dellsrvadmin_var_t;
files_type(dellsrvadmin_var_t)

domain_transition_pattern(udev_t, dellsrvadmin_exec_t, dellsrvadmin_t)
modutils_domtrans(dellsrvadmin_t)

allow dell_datamgrd_t device_t:dir rw_dir_perms;
allow dell_datamgrd_t device_t:chr_file create;

allow dell_datamgrd_t event_device_t:chr_file { read write };

allow dell_datamgrd_t self:process signal;
allow dell_datamgrd_t self:fifo_file rw_file_perms;
allow dell_datamgrd_t self:sem create_sem_perms;
allow dell_datamgrd_t self:capability { dac_override dac_read_search mknod sys_rawio sys_admin };
allow dell_datamgrd_t self:lockdown integrity;
allow dell_datamgrd_t self:unix_dgram_socket create_socket_perms;
allow dell_datamgrd_t self:netlink_route_socket r_netlink_socket_perms;

modutils_domtrans(dell_datamgrd_t)

can_exec(dell_datamgrd_t, dmidecode_exec_t)

allow dell_datamgrd_t dellsrvadmin_var_t:dir rw_dir_perms;
allow dell_datamgrd_t dellsrvadmin_var_t:file manage_file_perms;
allow dell_datamgrd_t dellsrvadmin_var_t:lnk_file read;
allow dell_datamgrd_t dellsrvadmin_var_t:sock_file manage_file_perms;

kernel_read_network_state(dell_datamgrd_t)
kernel_read_system_state(dell_datamgrd_t)

kernel_search_fs_sysctls(dell_datamgrd_t)
kernel_read_vm_overcommit_sysctl(dell_datamgrd_t)

# for /proc/bus/pci/*
kernel_write_proc_files(dell_datamgrd_t)

corecmd_exec_bin(dell_datamgrd_t)
corecmd_exec_shell(dell_datamgrd_t)
corecmd_shell_entry_type(dell_datamgrd_t)

dev_rx_raw_memory(dell_datamgrd_t)
dev_rw_generic_chr_files(dell_datamgrd_t)
dev_rw_ipmi_dev(dell_datamgrd_t)
dev_rw_sysfs(dell_datamgrd_t)

files_search_tmp(dell_datamgrd_t)

files_read_etc_files(dell_datamgrd_t)
files_read_etc_symlinks(dell_datamgrd_t)

files_read_usr_files(dell_datamgrd_t)

logging_search_logs(dell_datamgrd_t)

miscfiles_read_localization(dell_datamgrd_t)

storage_raw_read_fixed_disk(dell_datamgrd_t)
storage_raw_write_fixed_disk(dell_datamgrd_t)

can_exec(mon_local_test_t, dellsrvadmin_exec_t)
allow mon_local_test_t dellsrvadmin_var_t:dir search;
allow mon_local_test_t dellsrvadmin_var_t:file read_file_perms;
allow mon_local_test_t dellsrvadmin_var_t:file setattr;
allow mon_local_test_t dellsrvadmin_var_t:sock_file write;
allow mon_local_test_t dell_datamgrd_t:unix_stream_socket connectto;
allow mon_local_test_t self:sem { create read write destroy unix_write };

allow dellsrvadmin_t self:process signal;
allow dellsrvadmin_t self:lockdown integrity;
allow dellsrvadmin_t self:sem create_sem_perms;
allow dellsrvadmin_t self:fifo_file rw_file_perms;
allow dellsrvadmin_t self:packet_socket create;
allow dellsrvadmin_t self:unix_stream_socket { connectto create_stream_socket_perms };
allow dellsrvadmin_t self:capability { sys_admin sys_rawio };
dev_read_raw_memory(dellsrvadmin_t)
dev_rw_sysfs(dellsrvadmin_t)
dev_rx_raw_memory(dellsrvadmin_t)

allow dellsrvadmin_t dellsrvadmin_var_t:dir rw_dir_perms;
allow dellsrvadmin_t dellsrvadmin_var_t:file manage_file_perms;
allow dellsrvadmin_t dellsrvadmin_var_t:lnk_file read;
allow dellsrvadmin_t dellsrvadmin_var_t:sock_file write;

allow dellsrvadmin_t dell_datamgrd_t:unix_stream_socket connectto;

kernel_read_network_state(dellsrvadmin_t)
kernel_read_system_state(dellsrvadmin_t)

kernel_search_fs_sysctls(dellsrvadmin_t)
kernel_read_vm_overcommit_sysctl(dellsrvadmin_t)

corecmd_exec_bin(dellsrvadmin_t)
corecmd_exec_shell(dellsrvadmin_t)

corecmd_shell_entry_type(dellsrvadmin_t)

files_read_etc_files(dellsrvadmin_t)
files_read_etc_symlinks(dellsrvadmin_t)
files_read_usr_files(dellsrvadmin_t)

logging_search_logs(dellsrvadmin_t)
miscfiles_read_localization(dellsrvadmin_t)

Here is dellsrvadmin.fc:

/opt/dell/srvadmin/sbin/.*        --        gen_context(system_u:object_r:dellsrvadmin_exec_t,s0)
/opt/dell/srvadmin/sbin/dsm_sa_datamgrd        --        gen_context(system_u:object_r:dell_datamgrd_t,s0)
/opt/dell/srvadmin/bin/.*        --        gen_context(system_u:object_r:dellsrvadmin_exec_t,s0)
/opt/dell/srvadmin/var(/.*)?                        gen_context(system_u:object_r:dellsrvadmin_var_t,s0)
/opt/dell/srvadmin/etc/srvadmin-isvc/ini(/.*)?        gen_context(system_u:object_r:dellsrvadmin_var_t,s0)

21 July, 2024 08:57AM by etbe

July 19, 2024

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

dtts 0.1.3 on CRAN: More Maintenance

Leonardo and I are happy to announce the release of another maintenance release 0.1.3 of our dtts package which has been on CRAN for a good two years now.

dtts builds upon our nanotime package as well as the beloved data.table to bring high-performance and high-resolution indexing at the nanosecond level to data frames. dtts aims to offers the time-series indexing versatility of xts (and zoo) to the immense power of data.table while supporting highest nanosecond resolution.

This release contains two nice and focussed contributed pull requests. Tomas Kalibera, who as part of R Core looks after everything concerning R on Windows, and then some, needed an adjustment for pending / upcoming R on Windows changes for builds with LLVM which is what Arm-on-Windows uses. We happily obliged: neither Leonardo nor I see much of Windows these decades. (Easy thing to say on a day like today with its crowdstrike hammer falling!) Similarly, Michael Chirico supplied a PR updating one of our tests to an upcoming change at data.table which we are of course happy to support.

The short list of changes follows.

Changes in version 0.1.3 (2024-07-18)

  • Windows builds use localtime_s with LLVM (Tomas Kalibera in #16)

  • Tests code has been adjusted for an upstream change in data.table tests for all.equal (Michael Chirico in #18 addressing #17)

Courtesy of my CRANberries, there is also a report with diffstat for this release. Questions, comments, issue tickets can be brought to the GitHub repo. If you like this or other open-source work I do, you can now sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

19 July, 2024 08:49PM

hackergotchi for Bits from Debian

Bits from Debian

New Debian Developers and Maintainers (May and June 2024)

The following contributors got their Debian Developer accounts in the last two months:

  • Dennis van Dok (dvandok)
  • Peter Wienemann (wiene)
  • Quentin Lejard (valde)
  • Sven Geuer (sge)
  • Taavi Väänänen (taavi)
  • Hilmar Preusse (hille42)
  • Matthias Geiger (werdahias)
  • Yogeswaran Umasankar (yogu)

The following contributors were added as Debian Maintainers in the last two months:

  • Bernhard Miklautz
  • Felix Moessbauer
  • Maytham Alsudany
  • Aquila Macedo
  • David Lamparter
  • Tim Theisen
  • Stefano Brivio
  • Shengqi Chen

Congratulations!

19 July, 2024 02:00PM by Jean-Pierre Giraud

July 18, 2024

Enrico Zini

meson, includedir, and current directory

Suppose you have a meson project like this:

meson.build:

project('example', 'cpp', version: '1.0', license : '…', default_options: ['warning_level=everything', 'cpp_std=c++17'])

subdir('example')

example/meson.build:

test_example = executable('example-test', ['main.cc'])

example/string.h:

/* This file intentionally left empty */

example/main.cc:

#include <cstring>

int main(int argc,const char* argv[])
{
    std::string foo("foo");
    return 0;
}

This builds fine with autotools and cmake, but not meson:

$ meson setup builddir
The Meson build system
Version: 1.0.1
Source dir: /home/enrico/dev/deb/wobble-repr
Build dir: /home/enrico/dev/deb/wobble-repr/builddir
Build type: native build
Project name: example
Project version: 1.0
C++ compiler for the host machine: ccache c++ (gcc 12.2.0 "c++ (Debian 12.2.0-14) 12.2.0")
C++ linker for the host machine: c++ ld.bfd 2.40
Host machine cpu family: x86_64
Host machine cpu: x86_64
Build targets in project: 1

Found ninja-1.11.1 at /usr/bin/ninja
$ ninja -C builddir
ninja: Entering directory `builddir'
[1/2] Compiling C++ object example/example-test.p/main.cc.o
FAILED: example/example-test.p/main.cc.o
ccache c++ -Iexample/example-test.p -Iexample -I../example -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -Wcast-qual -Wconversion -Wfloat-equal -Wformat=2 -Winline -Wmissing-declarations -Wredundant-decls -Wshadow -Wundef -Wuninitialized -Wwrite-strings -Wdisabled-optimization -Wpacked -Wpadded -Wmultichar -Wswitch-default -Wswitch-enum -Wunused-macros -Wmissing-include-dirs -Wunsafe-loop-optimizations -Wstack-protector -Wstrict-overflow=5 -Warray-bounds=2 -Wlogical-op -Wstrict-aliasing=3 -Wvla -Wdouble-promotion -Wsuggest-attribute=const -Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wtrampolines -Wvector-operation-performance -Wsuggest-attribute=format -Wdate-time -Wformat-signedness -Wnormalized=nfc -Wduplicated-cond -Wnull-dereference -Wshift-negative-value -Wshift-overflow=2 -Wunused-const-variable=2 -Walloca -Walloc-zero -Wformat-overflow=2 -Wformat-truncation=2 -Wstringop-overflow=3 -Wduplicated-branches -Wattribute-alias=2 -Wcast-align=strict -Wsuggest-attribute=cold -Wsuggest-attribute=malloc -Wanalyzer-too-complex -Warith-conversion -Wbidi-chars=ucn -Wopenacc-parallelism -Wtrivial-auto-var-init -Wctor-dtor-privacy -Weffc++ -Wnon-virtual-dtor -Wold-style-cast -Woverloaded-virtual -Wsign-promo -Wstrict-null-sentinel -Wnoexcept -Wzero-as-null-pointer-constant -Wabi-tag -Wuseless-cast -Wconditionally-supported -Wsuggest-final-methods -Wsuggest-final-types -Wsuggest-override -Wmultiple-inheritance -Wplacement-new=2 -Wvirtual-inheritance -Waligned-new=all -Wnoexcept-type -Wregister -Wcatch-value=3 -Wextra-semi -Wdeprecated-copy-dtor -Wredundant-move -Wcomma-subscript -Wmismatched-tags -Wredundant-tags -Wvolatile -Wdeprecated-enum-enum-conversion -Wdeprecated-enum-float-conversion -Winvalid-imported-macros -std=c++17 -O0 -g -MD -MQ example/example-test.p/main.cc.o -MF example/example-test.p/main.cc.o.d -o example/example-test.p/main.cc.o -c ../example/main.cc
In file included from ../example/main.cc:1:
/usr/include/c++/12/cstring:77:11: error: memchr has not been declared in ::
   77 |   using ::memchr;
      |           ^~~~~~
/usr/include/c++/12/cstring:78:11: error: memcmp has not been declared in ::
   78 |   using ::memcmp;
      |           ^~~~~~
/usr/include/c++/12/cstring:79:11: error: memcpy has not been declared in ::
   79 |   using ::memcpy;
      |           ^~~~~~
/usr/include/c++/12/cstring:80:11: error: memmove has not been declared in ::
   80 |   using ::memmove;
      |           ^~~~~~~

It turns out that meson adds the current directory to the include path by default:

Another thing to note is that include_directories adds both the source directory and corresponding build directory to include path, so you don't have to care.

It seems that I have to care after all.

Thankfully there is an implicit_include_directories setting that can turn this off if needed.

Its documentation is not as easy to find as I'd like (kudos to Kangie on IRC), and hopefully this blog post will make it easier for me to find it in the future.

18 July, 2024 01:16PM

July 17, 2024

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

Rcpp 1.0.13 on CRAN: Some Updates

rcpp logo

The Rcpp Core Team is once again pleased to announce a new release (now at 1.0.13) of the Rcpp package. It arrived on CRAN earlier today, and has since been uploaded to Debian. Windows and macOS builds should appear at CRAN in the next few days, as will builds in different Linux distribution–and of course r2u should catch up tomorrow too. The release was uploaded last week, but not only does Rcpp always gets flagged because of the grandfathered .Call(symbol) but CRAN also found two packages ‘regressing’ which then required them to take five days to get back to us. One issue was known; another did not reproduce under our tests against over 2800 reverse dependencies leading to the eventual release today. Yay. Checks are good and appreciated, and it does take time by humans to review them.

This release continues with the six-months January-July cycle started with release 1.0.5 in July 2020. As a reminder, we do of course make interim snapshot ‘dev’ or ‘rc’ releases available via the Rcpp drat repo as well as the r-universe page and repo and strongly encourage their use and testing—I run my systems with these versions which tend to work just as well, and are also fully tested against all reverse-dependencies.

Rcpp has long established itself as the most popular way of enhancing R with C or C++ code. Right now, 2867 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 256 in BioConductor. On CRAN, 13.6% of all packages depend (directly) on Rcpp, and 59.9% of all compiled packages do. From the cloud mirror of CRAN (which is but a subset of all CRAN downloads), Rcpp has been downloaded 86.3 million times. The two published papers (also included in the package as preprint vignettes) have, respectively, 1848 (JSS, 2011) and 324 (TAS, 2018) citations, while the the book (Springer useR!, 2013) has another 641.

This release is incremental as usual, generally preserving existing capabilities faithfully while smoothing our corners and / or extending slightly, sometimes in response to changing and tightened demands from CRAN or R standards. The move towards a more standardized approach for the C API of R leads to a few changes; Kevin did most of the PRs for this. Andrew Johnsom also provided a very nice PR to update internals taking advantage of variadic templates.

The full list below details all changes, their respective PRs and, if applicable, issue tickets. Big thanks from all of us to all contributors!

Changes in Rcpp release version 1.0.13 (2024-07-11)

  • Changes in Rcpp API:

    • Set R_NO_REMAP if not already defined (Dirk in #1296)

    • Add variadic templates to be used instead of generated code (Andrew Johnson in #1303)

    • Count variables were switches to size_t to avoid warnings about conversion-narrowing (Dirk in #1307)

    • Rcpp now avoids the usage of the (non-API) DATAPTR function when accessing the contents of Rcpp Vector objects where possible. (Kevin in #1310)

    • Rcpp now emits an R warning on out-of-bounds Vector accesses. This may become an error in a future Rcpp release. (Kevin in #1310)

    • Switch VECTOR_PTR and STRING_PTR to new API-compliant RO variants (Kevin in #1317 fixing #1316)

  • Changes in Rcpp Deployment:

    • Small updates to the CI test containers have been made (#1304)

Thanks to my CRANberries, you can also look at a diff to the previous release Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page. Bugs reports are welcome at the GitHub issue tracker as well (where one can also search among open or closed issues).

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

17 July, 2024 09:50PM

Kalyani Kenekar

Securing Your Website: Installing and Configuring Nginx with SSL

Logo Nginx

The Initial Encounter:

I recently started to work with Nginx to explore the requirements on how to configure a then so called server block. It’s quite different than within Apache. But there are a tons of good websites out there which do explain the different steps and options quite well. I also realized quickly that I need to be able to configure my Nginx setups in a way so the content is delivered through https with some automatic redirection from http URLs.

  • Let’s install Nginx

Installing Nginx

$ sudo apt update
$ sudo apt install nginx

Checking your Web Server

  • We can check now nginx service is active or inactive
Output
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2024-02-12 09:59:20 UTC; 3h ago
       Docs: man:nginx(8)
   Main PID: 2887 (nginx)
      Tasks: 2 (limit: 1132)
     Memory: 4.2M
        CPU: 81ms
     CGroup: /system.slice/nginx.service
             ├─2887 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             └─2890 nginx: worker process
  • Now we successfully installed nginx and it in running state.

How To Secure Nginx with Let’s Encrypt on Debian 12

  • In this documentation, you will use Certbot to obtain a free SSL certificate for Nginx on Debian 12 and set up your certificate.

Step 1 — Installing Certbot

$ sudo apt install certbot python3-certbot-nginx

  • Certbot is now ready to use, but in order for it to automatically configure SSL for Nginx, we need to verify some of Nginx’s configuration.

Step 2 — Confirming Nginx’s Configuration

  • Certbot needs to be able to find the correct server block in your Nginx configuration for it to be able to automatically configure SSL. Specifically, it does this by looking for a server_name directive that matches the domain you request a certificate for. To check, open the configuration file for your domain using nano or your favorite text editor.

$ sudo vi /etc/nginx/sites-available/example.com

 server {
    listen 80;
    root /var/www/html/;
    index index.html;
    server_name example.com
    location / {
        try_files $uri $uri/ =404;
    }

    location /test.html {
        try_files $uri $uri/ =404;
        auth_basic "admin area";
        auth_basic_user_file /etc/nginx/.htpasswd;

    }
}
  • Fillup above data your project wise and then save the file, quit your editor, and verify the syntax of your configuration edits.

$ sudo nginx -t

Step 3 — Obtaining an SSL Certificate

  • Certbot provides a variety of ways to obtain SSL certificates through plugins. The Nginx plugin will take care of reconfiguring Nginx and reloading the config whenever necessary. To use this plugin, type the following command line.

$ sudo certbot --nginx -d example.com

  • The configuration will be updated, and Nginx will reload to pick up the new settings. certbot will wrap up with a message telling you the process was successful and where your certificates are stored.
Output
IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/example.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/example.com/privkey.pem
   Your cert will expire on 2024-05-12. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 - If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le
  • Your certificates are downloaded, installed, and loaded. Next check the syntax again of your configuration.

$ sudo nginx -t

  • If you get an error, reopen the server block file and check for any typos or missing characters. Once your configuration file’s syntax is correct, reload Nginx to load the new configuration.

$ sudo systemctl reload nginx

  • Try reloading your website using https:// and notice your browser’s security indicator. It should indicate that the site is properly secured, usually with a lock icon.

Now your website is secured by SSL usage.

17 July, 2024 06:30PM

hackergotchi for Gunnar Wolf

Gunnar Wolf

Script for weather reporting in Waybar

While I was living in Argentina, we (my family) found ourselves checking for weather forecasts almost constantly — weather there can be quite unexpected, much more so that here in Mexico. So it took me a bit of tinkering to come up with a couple of simple scripts to show the weather forecast as part of my Waybar setup. I haven’t cared to share with anybody, as I believe them to be quite trivial and quite dirty.

But today, Víctor was asking for some slightly-related things, so here I go. Please do remember I warned: Dirty.

Forecast

I am using OpenWeather’s open API. I had to register to get an APPID, and it allows me for up to 1,000 API calls per day, more than plenty for my uses, even if I am logged in at my desktops at three different computers (not an uncommon situation). Having that, I set up a file named /etc/get_weather/, that currently reads:

# Home, Mexico City
LAT=19.3364
LONG=-99.1819

# # Home, Paraná, Argentina
# LAT=-31.7208
# LONG=-60.5317

# # PKNU, Busan, South Korea
# LAT=35.1339
#LONG=129.1055

APPID=SomeLongRandomStringIAmNotSharing

Then, I have a simple script, /usr/local/bin/get_weather, that fetches the current weather and the forecast, and stores them as /run/weather.json and /run/forecast.json:

#!/usr/bin/bash
CONF_FILE=/etc/get_weather

if [ -e "$CONF_FILE" ]; then
    . "$CONF_FILE"
else
    echo "Configuration file $CONF_FILE not found"
    exit 1
fi

if [ -z "$LAT" -o -z "$LONG" -o -z "$APPID" ]; then
    echo "Configuration file must declare latitude (LAT), longitude (LONG) "
    echo "and app ID (APPID)."
    exit 1
fi

CURRENT=/run/weather.json
FORECAST=/run/forecast.json

wget -q "https://api.openweathermap.org/data/2.5/weather?lat=${LAT}&lon=${LONG}&units=metric&appid=${APPID}" -O "${CURRENT}"
wget -q "https://api.openweathermap.org/data/2.5/forecast?lat=${LAT}&lon=${LONG}&units=metric&appid=${APPID}" -O "${FORECAST}"

This script is called by the corresponding systemd service unit, found at /etc/systemd/system/get_weather.service:

[Unit]
Description=Get the current weather

[Service]
Type=oneshot
ExecStart=/usr/local/bin/get_weather

And it is run every 15 minutes via the following systemd timer unit, /etc/systemd/system/get_weather.timer:

[Unit]
Description=Get the current weather every 15 minutes

[Timer]
OnCalendar=*:00/15:00
Unit=get_weather.service

[Install]
WantedBy=multi-user.target

(yes, it runs even if I’m not logged in, wasting some of my free API calls… but within reason)

Then, I declare a "custom/weather" module in the desired position of my ~/.config/waybar/waybar.config, and define it as:

"custom/weather": {
    "exec": "while true;do /home/gwolf/bin/parse_weather.rb;sleep 10; done",
"return-type": "json",
},

This script basically morphs a generic weather JSON description into another set of JSON bits that display my weather in the way I prefer to have it displayed as:

#!/usr/bin/ruby
require 'json'

Sources = {:weather => '/run/weather.json',
           :forecast => '/run/forecast.json'
          }
Icons = {'01d' => '🌞', # d → day
         '01n' => '🌃', # n → night
         '02d' => '🌤️',
         '02n' => '🌥',
         '03d' => '☁️',
         '03n' => '🌤',
         '04d'  => '☁️',
         '04n' => '🌤',
         '09d' => '🌧️',
         '10n' =>  '🌧 ',
         '10d' => '🌦️',
         '13d' => '❄️',
         '50d' => '🌫️'
        }

ret = {'text': nil, 'tooltip': nil, 'class': 'weather', 'percentage': 100}

# Current weather report: Main text of the module
begin
  weather = JSON.parse(open(Sources[:weather],'r').read)

  loc_name = weather['name']
  icon = Icons[weather['weather'][0]['icon']] || '?' + f['weather'][0]['icon'] + f['weather'][0]['main']

  temp = weather['main']['temp']
  sens = weather['main']['feels_like']
  hum = weather['main']['humidity']

  wind_vel = weather['wind']['speed']
  wind_dir = weather['wind']['deg']

  portions = {}
  portions[:loc] = loc_name
  portions[:temp] = '%s 🌡%2.2f°C (%2.2f)' % [icon, temp, sens]
  portions[:hum] = '💧 %2d%%' % hum
  portions[:wind] = '🌬%2.2fm/s %d°' % [wind_vel, wind_dir]
  ret['text'] = [:loc, :temp, :hum, :wind].map {|p| portions[p]}.join(' ')
rescue => err
  ret['text'] = 'Could not process weather file (%s ⇒ %s: %s)' % [Sources[:weather], err.class, err.to_s]
end

# Weather prevision for the following hours/days
begin
  cast = []
  forecast = JSON.parse(open(Sources[:forecast], 'r').read)
  min = ''
  max = ''

  day=Time.now.strftime('%Y.%m.%d')
  by_day = {}
  forecast['list'].each_with_index do |f,i|
    by_day[day] ||= []
    time = Time.at(f['dt'])
    time_lbl = '%02d:%02d' % [time.hour, time.min]

    icon = Icons[f['weather'][0]['icon']] || '?' + f['weather'][0]['icon'] + f['weather'][0]['main']

    by_day[day] << f['main']['temp']
    if time.hour == 0
      min = '%2.2f' % by_day[day].min
      max = '%2.2f' % by_day[day].max
      cast << '        ↑ min: <b>%s°C</b> max: <b>%s°C</b>' % [min, max]
      day = time.strftime('%Y.%m.%d')
      cast << '     ┍━━━━━┫  <b>%04d.%02d.%02d</b> ┠━━━━━┑' %
              [time.year, time.month, time.day]
    end
    cast << '%s | %2.2f°C | 🌢%2d%% | %s %s' % [time_lbl,
                                                f['main']['temp'],
                                                f['main']['humidity'],
                                                icon,
                                                f['weather'][0]['description']
                                               ]
  end
  cast << '        ↑ min: <b>%s</b>°C max: <b>%s°C</b>' % [min, max]

  ret['tooltip'] = cast.join("\n")
  
rescue => err
  ret['tooltip'] = 'Could not process forecast file (%s ⇒ %s)' % [Sources[:forecast], err.class, err.to_s]
end


# Print out the result for Waybar to process
puts ret.to_json

The end result? Nothing too stunning, but definitively something I find useful and even nicely laid out:

Screenshot

Do note that it seems OpenWeather will return the name of the closest available meteorology station with (most?) recent data — for my home, I often get Ciudad Universitaria, but sometimes Coyoacán or even San Ángel Inn.

17 July, 2024 05:32PM

hackergotchi for Mike Gabriel

Mike Gabriel

Weather Experts with Translation Skills Needed!

Lomiri Weather App goes Open Meteo

In Ubuntu Touch / Lomiri, Maciej Sopyło has updated Lomiri's Weather App to operate against a different weather forecast provider (Open Meteo). Additionally, the new implementation is generic and pluggable, so other weather data providers can be added-in later.

Big thanks to Maciej for working on this just in time (the previous implementation's API has recently been EOL'ed and is not available anymore to Ubuntu Touch / Lomiri users).

Lomiri Weather App - new Meteorological Terms part of the App now

While the old weather data provider implementation obtained all the meteorological information as already localized strings from the provider, the new implementation requires all sorts of weather conditions being translated within the Lomiri Weather App itself.

The meteorological terms are probably not easy to translate for the usual software translator, so special help might be required here.

Call for Translations: Lomiri Weather App

So, if you feel entitled to help here, please join the Hosted Weblate service [1] and start working on Lomiri Weather App.

Thanks a lot!

light+love
Mike Gabriel (aka sunweaver)

[1] https://hosted.weblate.org/
[2] https://hosted.weblate.org/projects/lomiri/lomiri-weather-app/

17 July, 2024 10:05AM by sunweaver

Russell Coker

Samsung Galaxy Note 9 Review

After the VoLTE saga [1] and the problems with battery life on the PinePhonePro [2] (which lasted 4 hours while idle with the screen off in my last test a few weeks ago) I’m running a Galaxy Note 9 [3] with the default Samsung OS as my daily driver.

I don’t think that many people will be rushing out to buy a 2018 phone regardless of my review. For someone who wants a phone of such age (which has decent hardware and a low price) then good options are the Pixel phones which are all supported by LineageOS.

I recommend not buying this phone due to the fact that it doesn’t have support for VoLTE with LineageOS (and presumably any other non-Samsung Android build) and doesn’t have support from any other OS. The One Plus 6/6T has Mobian support [4] as well as LineageOS support and is worth considering.

The Note 9 still has capable hardware by today’s standards. A 6.4″ display is about as big as most people want in their pocket and 2960×1440 resolution in that size (516dpi) is probably as high as most people can see without a magnifying glass. The model I’m using has 8G of RAM which is as much as the laptop I was using at the start of this year. I don’t think that many people will have things that they actually want to do on a phone which needs more hardware than this. The only hardware feature in new phones which beats this is the large folding screen in some recent phones, but $2500+ (the price of such phones in Australia) is too much IMHO and the second hand market for folding phones is poor due to the apparently high incidence of screens breaking.

The Note 9 has the “Dex” environment for running as a laptop if you connect it to a USB-C dock. It can run nicely with a 4K monitor with USB keyboard and mouse. The UI is very similar to that of older versions of Windows.

The Samsung version of Android seems mostly less useful than the stock Google version or the LineageOS version. The Samsung keyboard flags words such as “gay” as spelling errors and it can’t be uninstalled even when you install a better keyboard app. There is a “Bixby” button on the side of the phone to launch the Bixby voice recognition app which can’t be mapped to any useful purpose, The Google keyboard has a voice dictation option which I will try out some time but that’s all I desire in terms of voice recognition. There are alerts about Samsung special deals and configuration options including something about signing in to some service and having it donate money to charity, I doubt that any users want such features. Apart from Dex the Samsung Android build is a good advert for LineageOS.

The screen has curved sides for no good reason. This makes it more difficult to make a protective phone case as a case can’t extend beyond the screen at the sides and therefore if it’s dropped and hits an edge (step, table, etc) then the glass can make direct contact with something. Also the curved sides reflect sunlight in all directions, this means that the user has to go to more effort to avoid reflecting the sun into their eyes and that a passenger can more easily reflect sunlight into the eyes of a car driver. It’s an impressive engineering feat to make a curved touch-screen but it doesn’t do any good for users.

The stylus is good as always and the screen is AMOLED so it doesn’t waste much power when in dark mode. There is a configuration option to display a clock all the time when the screen is locked because that apparently doesn’t use much power. I haven’t felt inclined to enable the always on screen but it’s a nice feature for those who like such things.

The VoLTE implementation is apparently a bit unusual so it’s not supported by LineageOS and didn’t work on Droidian for the small amount of time that Droidian supported it.

Generally this phone is quite nice hardware it’s just a pity that it demonstrates all of the downsides to buying a non-Pixel phone.

17 July, 2024 07:02AM by etbe

hackergotchi for Gunnar Wolf

Gunnar Wolf

Scholarly spam • «Wulfenia»

I just got one of those utterly funny spam messages… And yes, I recognize everybody likes building a name for themselves. But some spammers are downright silly.

I just got the following mail:

From: Hermine Wolf <hwolf850@gmail.com>
To: me, obviously 😉
Date: Mon, 15 Jul 2024 22:18:58 -0700
Subject: Make sure that your manuscript gets indexed and showcased in the prestigious Scopus database soon.
Message-ID: <CAEZZb3XCXSc_YOeR7KtnoSK4i3OhD=FH7u+A5xSMsYvhQZojQA@mail.gmail.com>

This message has visual elements included. If they don't display, please   
update your email preferences.

*Dear Esteemed Author,*


Upon careful examination of your recent research articles available online,
we are excited to invite you to submit your latest work to our esteemed    
journal, '*WULFENIA*'. Renowned for upholding high standards of excellence 
in peer-reviewed academic research spanning various fields, our journal is 
committed to promoting innovative ideas and driving advancements in        
theoretical and applied sciences, engineering, natural sciences, and social
sciences. 'WULFENIA' takes pride in its impressive 5-year impact factor of 
*1.000* and is highly respected in prestigious databases including the     
Science Citation Index Expanded (ISI Thomson Reuters), Index Copernicus,   
Elsevier BIOBASE, and BIOSIS Previews.                                     
                                                                           
*Wulfenia submission page:*                                                
[image: research--check.png][image: scrutiny-table-chat.png][image:        
exchange-check.png][image: interaction.png]                                
.                                                                          

Please don't reply to this email                                           
                                                                           
We sincerely value your consideration of 'WULFENIA' as a platform to       
present your scholarly work. We eagerly anticipate receiving your valuable 
contributions.                                                             

*Best regards,*                                                            
Professor Dr. Vienna S. Franz                                              

Scholarly spam

Who cares what Wulfenia is about? It’s about you, my stupid Wolf cousin!

17 July, 2024 12:23AM

July 16, 2024

Dave Hibberd

What I've been up to in Open Ham Radio - July 2024

I do a lot in free software for ham radio, and Steve at Zero Retries encouraged me to take this email I sent him and translate it into something here. UK Packet Radio Network UKPRN is going nicely, with the Nottingham and South segment really quite impressively interconnected over RF - https://nodes.ukpacketradio.network/packet-network-map.html?rfonly=1 I’m excited to see the growth down there! We’re sorting out forwarding and routes in Aberdeen too, and working to grow the RF path to Inverness.

16 July, 2024 06:10PM

hackergotchi for Bits from Debian

Bits from Debian

Wind River Platinum Sponsor of DebConf24

windriverlogo

We are pleased to announce that Wind River has committed to sponsor DebConf24 as a Platinum Sponsor.

For nearly 20 years, Wind River has led in commercial open source Linux solutions for mission-critical enterprise edge computing. With expertise across aerospace, automotive, industrial, telecom, more, the company is committed to open source through initiatives like eLxr, Yocto, Zephyr, and StarlingX.

With this commitment as Platinum Sponsor, Wind River is contributing to make possible our annual conference, and directly supporting the progress of Debian and Free Software, helping to strengthen the community that continues to collaborate on Debian projects throughout the rest of the year.

Wind River plans to announce an exiting new project based on Debian at this year's DebConf!

Thank you very much, Wind River, for your support of DebConf24!

Become a sponsor too!

DebConf24 will take place from 28th July to 4th August 2024 in Busan, South Korea, and will be preceded by DebCamp, from 21st to 27th July 2024.

DebConf24 is accepting sponsors! Interested companies and organizations should contact the DebConf team through sponsors@debconf.org, or visit the DebConf24 website at https://debconf24.debconf.org/sponsors/become-a-sponsor/.

16 July, 2024 03:08PM by Sahil Dhiman

July 15, 2024

hackergotchi for Steinar H. Gunderson

Steinar H. Gunderson

Pull requests via git push

This project inspired me to investigate whether git.sesse.net could start accepting patches in a format that was less friction than email, and didn't depend on custom SSH-facing code written by others. And it seems it really can! The thought was to simply allow git push from anyone, but that git push doesn't actually push anything; it just creates a pull request (by email). It was much simpler than I'd thought. First make an empty hooks directory with this pre-receive hook (make sure it is readable by your web server, and marked as executable):

#! /bin/bash
set -e
read oldsha newsha refname
git send-email --to=steinar+git@gunderson.no --suppress-cc=all --subject-prefix="git-anon-push PATCH" --quiet $oldsha..$newsha
echo ''
echo 'Thank you for your contribution! The patch has been sent by email and will be examined for inclusion.'
echo 'The push will now exit with an error. No commits have actually been pushed.'
exit 1

Now we can activate this hook and anonymous push in each project (I already run git-http-backend on the server for pulling, and it supports just fine if you tell it to), and give www-data write permissions to store the pushed objects temporarily:

git config core.hooksPath /srv/git.sesse.net/hooks
git config http.receivepack true
sudo chgrp -R www-data .
chmod -R g+w .

And now any attempts to git push will send me patch emails that I can review and optionally include!

It's not perfect. For instance, it doesn't support multipush, and if you try to push to a branch that doesn't exist already, will error out since $oldsha is all-zeros. And the From: header is always www-data (but I didn't want to expose myself to all sorts of weird injection attacks by trying to parse the committer email). And of course, there's no spam control, but if you want to spam me with email, then you could just like… send email?

(I have backups, in case someone discovers some sort of evil security hole.)

15 July, 2024 11:15AM

hackergotchi for Thomas Lange

Thomas Lange

FAIme adds Korean language support

In two weeks DebConf24, the Debian conference starts in Busan, South Korea. Therefore I've added support for the Korean language into the web service of FAI:

https://fai-project.org/FAIme/

Another new feature of the FAIme service will be announced at DebConf24 in August.

15 July, 2024 11:01AM

July 14, 2024

Russ Allbery

podlators v6.0.2

podlators contains the Perl modules and scripts used to convert Perl's documentation language, POD, to text and manual pages.

This is another small bug fix release that is part of iterating on getting the new podlators incorproated into Perl core. The bug fixed in this release was another build system bug I introduced in recent refactorings, this time breaking the realclean target so that some generated scripts were not removed. Thanks to James E Keenan for the report.

You can get the latest version from CPAN or from the podlators distribution page.

14 July, 2024 07:53PM

DocKnot 8.0.1

DocKnot is my static web site generator, with some additional features for managing software releases.

This release fixes some bugs in the newly-added conversion of text to HTML that were due to my still-incomplete refactoring of that code. It still uses some global variables, and they were leaking between different documents and breaking the formatting. It also fixes consistency problems with how the style parameter in *.spin files was interpreted, and fixes some incorrect docknot update-spin behavior.

You can get the latest version from CPAN or from the DocKnot distribution page.

14 July, 2024 07:38PM

Ravi Dwivedi

Kenya Visa Process

Prior to arrival in Kenya, you need to apply for an Electronic Travel Authorization (eTA) on their website by uploading all the required documents. This system is in place since Jan 2024 after the country abolished the visa system, implementing the eTA portal. The required documents will depend on the purpose of your visit, which in my case, was to attend a conference.

Here is the list of documents I submitted for my eTA:

  • Scanned copy of my passport

  • Photograph with white background

  • Flight tickets (reservation)

  • Hotel bookings (reservation)

  • Invitation letter from the conference

  • Yellow Fever vaccination certificate (optional)

  • Job contract (optional)

“Reservation” means I didn’t book the flights and hotels, but rather reserved them. Additionally, “optional” means that those documents were not mandatory to submit, but I submitted them in the “Other Documents” section in order to support my application. After submitting the eTA, I had to make a payment of around 35 US Dollars (approximately 3000 Indian Rupees).

It took 40 hours for me to receive an email from Kenya stating that my eTA has been approved, along with an attached PDF, making this one of my smoothest experiences of obtaining travel documents to travel to a country :). An eTA is technically not a visa, but I put the word “visa” in the title due to familiarity with the term.

14 July, 2024 10:54AM

July 13, 2024

Anuradha Weeraman

Windows of Opportunity: Microsoft's Open Source Renaissance

Windows of Opportunity: Microsoft's Open Source Renaissance

Twenty years ago, it was easy to dislike Microsoft. It was the quintessential evil MegaCorp that was quick to squash competition, often ruthlessly, but in some cases slowly through a more insidious process of embracing, extending, and exterminating anything that got in the way. This was the signature personality of Ballmer-era Microsoft that also inspired and united the software freedom fighting forces that came together to safeguard things that mattered to them and were at risk.

I remember the era when the Novell, SCO, and Microsoft saga cast fear, uncertainty, and doubt on the future of open Unix and Linux and on what would happen to the operating systems that we loved if the suits of Redmond prevailed. Looking back, I&aposm glad that the arc of this story has bent towards justice, and I shudder at the possibilities had it worked out differently.

Looking at today&aposs Microsoft, I&aposm amazed at how much change a leader with the right vision can make to the trajectory of a company that even makes an old-school software freedom advocate as me admire and even applaud the strides it has taken in the last 10 or so years that has dramatically shifted the perception of Microsoft. The personality of the Satya-era Microsoft is one to behold. While it will take more time to win back the trust, we see the tides changing and the positivity is important for the entire industry.

For Microsoft, it was TypeScript and VS Code that helped change the narrative internally which led to its internal resurgence and acceptance of open source. Its acquisition of GitHub propelled it forward within the community overnight. Its contributions to the Linux kernel and other major software projects have also been consequential in changing its public perceptions.

It takes a while to claw back trust and is very easy to breach. This time, however, Microsoft seems to understand this dynamic more than it did 20 years ago. All it took was the right leadership.

13 July, 2024 01:40PM by Anuradha Weeraman

Ravi Dwivedi

Yellow Fever Vaccine

Recently, I got vaccinated with yellow fever vaccine as I am planning to travel to Kenya, a high risk country for yellow fever, in the near future. The vaccine takes 10 days to produce the required antibodies, so it should be taken at least 10 days before the date of departure to the at-risk country. In order to get vaccinated, I searched for vaccination centers in Delhi for yellow fever. I found this page by the Indian government which lists vaccination centers for yellow fever all over India. From that list, I made a phone call to the Airport Health Organization, a vaccination center near to the Delhi Airport.

They asked me to write an email stating that I need yellow fever vaccination. After sending the email, they requested a scanned copy of my passport. Subsequently, they emailed me my appointment date, asking me to pay 300 INR in advance along with other instructions.

You have to reach vaccination center at any time between 10 AM to 12 noon. I got there at around 11 AM on my appointment date and got vaccinated in around 40 minutes, followed by obtaining a vaccine certificate in half an hour. One dosage of this vaccine gives immunity against yellow fever for lifetime. Therefore, I can travel to any country at risk of yellow fever. Although some countries may require proof of vaccination within some time frame and some people might need a booster dose to maintain immunity.

13 July, 2024 07:56AM

Russ Allbery

podlators v6.0.1

This is a small bug-fix release to remove use of autodie from the build system for the module. podlators is included in Perl core, and at the point when it is built during the core build, the prerequisites of the autodie module are not yet met, so the module is not available. This release reverts to explicit error checking in all the files used by the build system.

Thanks to James E Keenan for the report and the analysis.

You can get the latest version from CPAN or the podlators distribution page.

13 July, 2024 04:18AM

July 12, 2024

Reproducible Builds

Reproducible Builds in June 2024

Welcome to the June 2024 report from the Reproducible Builds project!

In our reports, we outline what we’ve been up to over the past month and highlight news items in software supply-chain security more broadly. As always, if you are interested in contributing to the project, please visit our Contribute page on our website.

Table of contents:

  1. Next Reproducible Builds Summit dates announced
  2. GNU Guix patch review session for reproducibility
  3. New reproducibility-related academic papers
  4. Misc development news
  5. Website updates
  6. Reproducibility testing framework


Next Reproducible Builds Summit dates announced

We are very pleased to announce the upcoming Reproducible Builds Summit, set to take place from September 17th — 19th 2024 in Hamburg, Germany.

We are thrilled to host the seventh edition of this exciting event, following the success of previous summits in various iconic locations around the world, including Venice, Marrakesh, Paris, Berlin and Athens. Our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field. Our aim is to create an inclusive space that fosters collaboration, innovation and problem-solving.

If you’re interesting in joining us this year, please make sure to read the event page which has more details about the event and location. We are very much looking forward to seeing many readers of these reports there.


GNU Guix patch review session for reproducibility

Vagrant Cascadian will holding a Reproducible Builds session as part of the monthly Guix patch review series on July 11th at 17:00 UTC.

These online events are intended to encourage everyone everyone becoming a patch reviewer and the goal of reviewing patches is to help Guix project accept contributions while maintaining our quality standards and learning how to do patch reviews together in a friendly hacking session.


Multiple scholarly papers related to Reproducible Builds were published this month:

An Industry Interview Study of Software Signing for Supply Chain Security was published by Kelechi G. Kalu, Tanmay Singla, Chinenye Okafor, Santiago Torres-Arias and James C. Davis of Electrical and Computer Engineering department of Purdue University, Indiana, USA, and is concerned with:

To understand software signing in practice, we interviewed 18 high-ranking industry practitioners across 13 organizations. We provide possible impacts of experienced software supply chain failures, security standards, and regulations on software signing adoption. We also study the challenges that affect an effective software signing implementation.


DiVerify: Diversifying Identity Verification in Next-Generation Software Signing was written by Chinenye L. Okafor, James C. Davis and Santiago Torres-Arias also of Purdue University and is interested in:

Code signing enables software developers to digitally sign their code using cryptographic keys, thereby associating the code to their identity. This allows users to verify the authenticity and integrity of the software, ensuring it has not been tampered with. Next-generation software signing such as Sigstore and OpenPubKey simplify code signing by providing streamlined mechanisms to verify and link signer identities to the public key. However, their designs have vulnerabilities: reliance on an identity provider introduces a single point of failure, and the failure to follow the principle of least privilege on the client side increases security risks. We introduce Diverse Identity Verification (DiVerify) scheme, which strengthens the security guarantees of next-generation software signing by leveraging threshold identity validations and scope mechanisms.


Felix Lagnöhed published their thesis on the Integration of Reproducibility Verification with Diffoscope in GNU Make. This work, amongst some other results:

[…] resulted in an extension of GNU make which is called rmake, where diffoscope — a tool for detecting differences between a large number of file types — was integrated into the workflow of make. rmake was later used to answer the posed research questions for this thesis. We found that different build paths and offsets are a big problem as three out of three tested Free and Open Source Software projects all contained these variations. The results also showed that gcc’s optimisation levels did not affect reproducibility, but link-time optimisation embeds a lot of unreproducible information in build artefacts. Lastly, the results showed that build paths, build ID’s and randomness are the three most common groups of variations encountered in the wild and potential solutions for some variations were proposed.


Lastly, Pol Dellaiera completed his master thesis on Reproducibility in Software Engineering at the University of Mons, Belgium, under the supervision of Dr. Tom Mens, professor and director of the Software Engineering Lab.

The thesis serves as an introduction to the concept of reproducibility in software engineering, offering a comprehensive overview of formalizations using mathematical notations for key concepts and an empirical evaluation of several key tools. By exploring various case studies, methodologies and tools, the research aims to provide actionable insights for practitioners and researchers alike.


Development news

In Debian this month, 4 reviews of Debian packages were added, 11 were updated and 14 were removed this month adding to our knowledge about identified issues. Only one issue types was updated, though, explaining that we don’t vary the build path anymore.


On our mailing list this month, Bernhard M. Wiedemann wrote that whilst he had previously collected issues that introduce non-determinism he has now moved on to discuss about “mitigations”, in the sense of how can we avoid whole categories of problem “without patching an infinite number of individual packages”. In addition, Janneke Nieuwenhuizen announced the release of two versions of GNU Mes. [][]


In openSUSE news, Bernhard M. Wiedemann published another report for that distribution.


In NixOS, with the 24.05 release out, it was again validated that our minimal ISO is reproducible by building it on a virtual machine with no access to the binary cache.


What’s more, we continued to write patches in order to fix specific reproducibility issues, including Bernhard M. Wiedemann writing three patches (for qutebrowser, samba and systemd), Chris Lamb filing Debian bug #1074214 against the fastfetch package and Arnout Engelen proposing fixes to refind and for the Scala compiler [.


Lastly, diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb uploaded two versions (270 and 271) to Debian, and made the following changes as well:

  • Drop Build-Depends on liblz4-tool in order to fix Debian bug #1072575. []
  • Update tests to support zipdetails version 4.004 that is shipped with Perl 5.40. []


Website updates

There were a number of improvements made to our website this month, including Akihiro Suda very helpfully making the <h4> elements more distinguishable from the <h3> level [][] as well as adding a guide for Dockerfile reproducibility []. In addition Fay Stegerman added two tools, apksigcopier and reproducible-apk-tools, to our Tools page.


Reproducibility testing framework

The Reproducible Builds project operates a comprehensive testing framework running primarily at tests.reproducible-builds.org in order to check packages and other artifacts for reproducibility. In June, a number of changes were made by Holger Levsen, including:

  • Marking the virt(32|64)c-armhf nodes as down. []
  • Granting a developer access to the osuosl4 node in order to debug a regression on the ppc64el architecture. []
  • Granting a developer access to the osuosl4 node. [][]

In addition, Mattia Rizzolo re-aligned the /etc/default/jenkins file with changes performed upstream [] and changed how configuration files are handled on the rb-mail1 host. [], whilst Vagrant Cascadian documented the failure of the virt32c and virt64c nodes after initial investigation [].



If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

12 July, 2024 12:32PM

Russ Allbery

Review: The Splinter in the Sky

Review: The Splinter in the Sky, by Kemi Ashing-Giwa

Publisher: Saga Press
Copyright: July 2023
ISBN: 1-6680-0849-1
Format: Kindle
Pages: 372

The Splinter in the Sky is a stand-alone science fiction political thriller. It is Kemi Ashing-Giwa's first novel.

Enitan is from Koriko, a vegetation-heavy moon colonized by the Vaalbaran empire. She lives in the Ijebu community with her sibling Xiang and has an on-again, off-again relationship with Ajana, the Vaalbaran-appointed governor. Xiang is studying to be an architect, which requires passing stringent entrance exams to be allowed to attend an ancillary imperial school intended for "primitives." Enitan works as a scribe and translator, one of the few Korikese allowed to use the sacred Orin language of Vaalbara. In her free time, she grows and processes tea.

When Xiang mysteriously disappears while she's at work, Enitan goes to Ajana for help. Then Ajana dies, supposedly from suicide. The Vaalbaran government demands a local hostage while the death is investigated, someone who will be held as a diplomatic "guest" on the home world and executed if there is any local unrest. This hostage is supposed to be the child of the local headwoman, a concept that the Korikese do not have. Seeing a chance to search for Xiang, Enitan volunteers, heading into the heart of imperial power with nothing but desperate determination and a tea set.

The empire doesn't stand a chance.

Admittedly, a lot of the reason why the empire doesn't stand a chance is because the author is thoroughly on Enitan's side. Before she even arrives on Gondwana, Vaalbara's home world, Enitan is recruited as a spy by the other Gondwana power and Vaalbara's long-standing enemy. Her arrival in the Splinter, the floating arcology that serves as the center of Vaalbaran government, is followed by a startlingly meteoric rise in access. Some of this is explained by being a cultural curiosity for bored nobles, and some is explained by political factors Enitan is not yet aware of, but one can see the author's thumb resting on the scales.

This was the sort of book that was great fun to read, but whose political implausibility provoked "wait, that didn't make sense" thoughts afterwards. I think one has to assume that the total population of Vaalbara is much less than first comes to mind when considering an interplanetary empire, which would help explain the odd lack of bureaucracy. Enitan is also living in, effectively, the palace complex, for reasonably well-explained political reasons, and that could grant her a surprising amount of access. But there are other things that are harder to explain away: the lack of surveillance, the relative lack of guards, and the odd political structure that's required for the plot to work.

It's tricky to talk about this without spoilers, but the plot rests heavily on a conspiratorial view of how government power is wielded that I think strains plausibility. I'm not naive enough to think that the true power structure of a society matches the formal power structure, but I don't think they diverge as much as people think they do. It's one thing to say that the true power brokers of society can be largely unknown to the general population. In a repressive society with a weak media, that's believable. It's quite another matter for the people inside the palace to be in the dark about who is running what.

I thought that was the biggest problem with this book. Its greatest feature is the characters, and particularly the character relationships. Enitan is an excellent protagonist: fascinating, sympathetic, determined, and daring in ways that make her success more believable. Early in the book, she forms an uneasy partnership that becomes the heart of the book, and I loved everything about that relationship. The politics of her situation might be a bit too simple, but the emotions were extremely well-done.

This is a book about colonialism. Specifically, it's a book about cultural looting, appropriation, and racist superiority. The Vaalbarans consider Enitan barely better than an animal, and in her home they're merciless and repressive. Taken out of that context into their imperial capital, they see her as a harmless curiosity and novelty. Enitan exploits this in ways that are entirely believable. She is also driven to incandescent fury in ways that are entirely believable, and which she only rarely can allow herself to act on. Ashing-Giwa drives home the sheer uselessness of even the more sympathetic Vaalbarans more forthrightly than science fiction is usually willing to be. It's not a subtle point, but it is an accurate one.

The first two thirds of this book had me thoroughly engrossed and unable to put it down. The last third unfortunately turns into a Pokémon hunt of antagonists, which I found less satisfying and somewhat less believable. I wish there had been more need for Enitan to build political alliances and go deeper into the social maneuverings of the first part of the book, rather than gaining some deus ex machina allies who trivially solve some otherwise-tricky plot problems. The setup is amazing; the resolution felt a bit like escaping a maze by blasting through the walls, which I don't think played to the strengths of the characters and relationships that Ashing-Giwa had constructed. The advantage of that approach is that we do get a satisfying resolution and a standalone novel.

The central relationship of the book is unfortunately too much of a spoiler to talk about in a review, but I thought it was the best part of the story. This is a political thriller on the surface, but I think it's heart is an unexpected political alliance with a fascinatingly tricky balance of power. I was delighted that Ashing-Giwa never allows the tension in that relationship to collapse into one of the stock patterns it so easily could have become.

The Splinter in the Sky reminded me a little of Arkady Martine's A Memory Called Empire. It's not as assured or as adroitly balanced as that book, and the characters are not quite as memorable, but that's a very high bar. The political point is even sharper, and it has some of the same appeal.

I had so much fun reading this book. You may need to suspend your disbelief about some of the politics, and I wish the conclusion had been a bit less brute-force, but this is great stuff. Recommended when you're in the mood for a character story in the trappings of a political thriller.

Rating: 8 out of 10

12 July, 2024 03:28AM

hackergotchi for Freexian Collaborators

Freexian Collaborators

Monthly report about Debian Long Term Support, June 2024 (by Roberto C. Sánchez)

Like each month, have a look at the work funded by Freexian’s Debian LTS offering.

Debian LTS contributors

In June, 18 contributors have been paid to work on Debian LTS, their reports are available:

  • Adrian Bunk did 47.0h (out of 74.25h assigned and 11.75h from previous period), thus carrying over 39.0h to the next month.
  • Arturo Borrero Gonzalez did 6.0h (out of 6.0h assigned).
  • Bastien Roucariès did 20.0h (out of 20.0h assigned).
  • Ben Hutchings did 15.5h (out of 16.0h assigned and 8.0h from previous period), thus carrying over 8.5h to the next month.
  • Chris Lamb did 18.0h (out of 18.0h assigned).
  • Daniel Leidert did 4.0h (out of 8.0h assigned and 2.0h from previous period), thus carrying over 6.0h to the next month.
  • Emilio Pozuelo Monfort did 23.25h (out of 49.5h assigned and 10.5h from previous period), thus carrying over 36.75h to the next month.
  • Guilhem Moulin did 4.5h (out of 13.0h assigned and 7.0h from previous period), thus carrying over 15.5h to the next month.
  • Lee Garrett did 17.0h (out of 25.0h assigned and 35.0h from previous period), thus carrying over 43.0h to the next month.
  • Lucas Kanashiro did 5.0h (out of 10.0h assigned and 10.0h from previous period), thus carrying over 15.0h to the next month.
  • Markus Koschany did 40.0h (out of 40.0h assigned).
  • Ola Lundqvist did 10.0h (out of 6.5h assigned and 17.5h from previous period), thus carrying over 14.0h to the next month.
  • Roberto C. Sánchez did 5.25h (out of 7.75h assigned and 4.25h from previous period), thus carrying over 6.75h to the next month.
  • Santiago Ruano Rincón did 22.5h (out of 14.5h assigned and 8.0h from previous period).
  • Sean Whitton did 6.5h (out of 6.0h assigned and 0.5h from previous period).
  • Stefano Rivera did 0.5h (out of 0.0h assigned and 10.0h from previous period), thus carrying over 9.5h to the next month.
  • Sylvain Beucler did 9.0h (out of 24.5h assigned and 35.5h from previous period), thus carrying over 51.0h to the next month.
  • Thorsten Alteholz did 14.0h (out of 14.0h assigned).

Evolution of the situation

In June, we have released 31 DLAs.

Notable security updates in June included:

  • git: multiple vulnerabilities, which may result in privilege escalation, denial of service, and arbitrary code execution
  • sendmail: SMTP smuggling allowed remote attackers bypass SPF protection checks
  • cups: arbitrary remote code execution

Looking further afield to the broader Debian ecosystem, LTS contributor Bastien Roucariès also patched sendmail in Debian 12 (bookworm) and 11 (bullseye) in order to fix the previously mentioned SMTP smuggling vulnerability. Furthermore, LTS contributor Thorsten Alteholz provided fixes for the cups packages in Debian 12 (bookworm) and 11 (bullseye) in order to fix the aforementioned arbitrary remote code execution vulnerability.

Additionally, LTS contributor Ben Hutchings has commenced work on an updated backport of Linux kernel 6.1 to Debian 11 (bullseye), in preparation for bullseye transitioning to the responsibility of the LTS (and the associated closure of the bullseye-backports repository). LTS Lucas Kanashiro also began the preparatory work of backporting parts of the rust/cargo toolchain to Debian 11 (bullseye) in order to make future updates of the clamav virus scanner possible.

June was the final month of LTS for Debian 10 (as announced on the debian-lts-announce mailing list). No additional Debian 10 security updates will be made available on security.debian.org.

However, Freexian and its team of paid Debian contributors will continue to maintain Debian 10 going forward for the customers of the Extended LTS offer. Subscribe right away if you sill have Debian 10 which must be kept secure (and which cannot yet be upgraded).

Thanks to our sponsors

Sponsors that joined recently are in bold.

12 July, 2024 12:00AM by Roberto C. Sánchez

July 11, 2024

Petter Reinholdtsen

More than 200 orphaned Debian packages moved to git, 216 to go

In April, I started migrating orphaned Debian packages without any version control system listed in debian/control to git. This morning, my Debian QA page finally reached 200 QA packages migrated. In reality there are a few more, as the packages uploaded by someone else after my initial upload have disappeared from my QA uploads list. As I am running out of steam and will most likely focus on other parts of Debian moving forward, I hope someone else will find time to continue the migration to bring the number of orphaned packages without any version control system down to zero. Here is the updated recipe if someone want to help out.

To locate packages to work on, the following one-liner can be used:

PGPASSWORD="udd-mirror" psql --port=5432 --host=udd-mirror.debian.net \
  --username=udd-mirror udd -c "select source from sources \
   where release = 'sid' and (vcs_url ilike '%anonscm.debian.org%' \
   OR vcs_browser ilike '%anonscm.debian.org%' or vcs_url IS NULL \
   OR vcs_browser IS NULL) AND maintainer ilike '%packages@qa.debian.org%' \
   order by random() limit 10;"

Pick a random package from the list and run the latest edition of the script debian-snap-to-salsa with the package name as the argument to prepare a git repository with the existing packaging. This will download old Debian packages from snapshot.debian.org. Note that very recent uploads will not be included, so check out the package on tracker.debian.org. Next, run gbp buildpackage --git-ignore-new to verify that the package build as it should, and then visit https://salsa.debian.org/debian/ and make sure there is not already a git repository for the package there. I also did git log -p debian/control and look for vcs entries to check if the package used to have a git repository on Alioth, and see if it can be a useful starting point moving forward. If all this check out, I created a new gitlab project below the Debian group on salsa, push the package source there and upload a new version. I tend to also ensure build hardening is enabled, if it prove to be easy, and check if I can easily fix any lintian issues or bug reports. If the process took more than 20 minutes, I dropped it and moved on to another package.

If I found patches in debian/patches/ that were not yet passed upstream, I would send an email to make sure upstream know about them. This has proved to be a valuable step, and caused several new releases for software that initially appeared abandoned. :)

As usual, if you use Bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

11 July, 2024 10:30AM

Russ Allbery

podlators v6.0.0

podlators is the collection of Perl modules and front-end scripts that convert POD documentation to *roff manual pages or text, possibly with formatting intended for pagers.

This release continues the simplifications that I've been doing in the last few releases and now uniformly escapes - characters and single quotes, disabling all special interpretation by *roff formatters and dropping the heuristics that were used in previous versions to try to choose between possible interpretations of those characters. I've come around to the position that POD simply is not semantically rich enough to provide sufficient information to successfully make a more nuanced conversion, and these days Unicode characters are available for authors who want to be more precise.

This version also drops support for Perl versions prior to 5.12 and switches to semantic versioning for all modules. I've added a v prefix to the version number, since that is the convention for Perl module versions that use semantic versioning.

This release also works around some changes to the man macros in groff 1.23.0 to force persistent ragged-right justification when formatted with nroff and fixes a variety of other bugs.

You can get the latest release from CPAN or from the podlators distribution page.

11 July, 2024 02:57AM

July 10, 2024

Petter Reinholdtsen

Some notes from the 2024 LinuxCNC Norwegian developer gathering

The Norwegian The LinuxCNC developer gathering 2024 is over. It was a great and productive weekend, and I am sad that it is over.

Regular readers probably still remember what LinuxCNC is, but her is a quick summary for those that forgot? LinuxCNC is a free software system for numerical control of machines such as milling machines, lathes, plasma cutters, routers, cutting machines, robots and hexapods. It eats G-code and produce motor movement and other changes to the physical world, while reading sensor input.

I am not quite sure about the total head count, as not all people were present at the gathering the entire weekend, but I believe it was close to 10 people showing their faces at the gathering. The "hard core" of the group, who stayed the entire weekend, were two from Norway, two from Germany and one from England. I am happy with the outcome from the gathering. We managed to wrap up a new stable LinuxCNC release 2.9.3 and even tested it on real hardware within minutes of the release. The release notes for 2.9.3 are still being written, but should show up on on the project site in the next few days. We managed to go through around twenty pull requests and merge then into either the stable release (2.9) or the development branch (master). There are still around thirty pull requests left to process, so we are not out of work yet. We even managed to fix/improve a slightly worn lathe, and experiment with running a mechanical clock using G-code.

The evening barbeque worked well both on Saturday and Sunday. It is quite fun to light up a charcoal grill using compressed air. Sadly the weather was not the best, so we stayed indoors most of the time.

This gathering was made possible partly with sponsoring from both Redpill Linpro, Debian and NUUG Foundation, and we are most grateful for the support. I would also like to thank the local school for lending us some furniture, and of course the rest of the members of the organizers team, Asle and Bosse, for their countless contributions. The gathering was such success that we want to do it again next year.

We plan to organize the next Norwegian LinuxCNC developer gathering at the end of June next year, the weekend Friday 27th to Sunday 29th of June 2025. I recommend you reserve the dates on your calendar today. Other related communities are also welcome to join in, for example those working on systems like FreeCAD and opencamlib, as I am sure we have much in common and sharing experiences would be very useful to all involved. We are of course looking for sponsors for this gathering already. The total budget for this gathering was around NOK 25.000 (around EUR 2.300), so our needs are quite modest. Perhaps a machine or tools company would like to help out the free software manufacturing community by sponsoring food, lodging and transport for such gathering?

10 July, 2024 12:45PM

Russell Coker

Computer Adavances in the Last Decade

I wrote a comment on a social media post where someone claimed that there’s no computer advances in the last 12 years which got long so it’s worth a blog post.

In the last decade or so new laptops have become cheaper than new desktop PCs. USB-C has taken over for phones and for laptop charging so all recent laptops support USB-C docks and monitors with USB-C docks built in have become common. 4K monitors have become cheap and common and higher than 4K is cheap for some use cases such as ultra wide. 4K TVs are cheap and TVs with built-in Android computers for playing internet content are now standard. For most use cases spinning media hard drives are obsolete, SSDs large enough for all the content most people need to store are cheap. We have gone from gigabit Ethernet being expensive to 2.5 gigabit being cheap.

12 years ago smart phones were very limited and every couple of years there would be significant improvements. Since about 2018 phones have been capable of doing most things most people want. 5yo Android phones can run the latest apps and take high quality pics. Any phone that supports VoLTE will be good for another 5+ years if it has security support. Phones without security support still work and are quite usable apart from being insecure. Google and Samsung have significantly increased their minimum security support for their phones and the GKI project from Google makes it easier for smaller vendors to give longer security support. There are a variety of open Android projects like LineageOS which give longer security support on a variety of phones. If you deliberately choose a phone that is likely to be well supported by projects like LineageOS (which pretty much means just Pixel phones) then you can expect to be able to actually use it when it is 10 years old. Compare this to the Samsung Galaxy S3 released in 2012 which was a massive improvement over the original Galaxy S (the S2 felt closer to the S than the S3). The Samsung Galaxy S4 released in 2013 was one of the first phones to have FullHD resolution which is high enough that most people can’t easily recognise the benefits of higher resolution. It wasn’t until 2015 that phones with 4G of RAM became common which is enough that for most phone use it’s adequate today.

Now that 16G of RAM is affordable in laptops running more secure OSs like Qubes is viable for more people. Even without Qubes, OS security has been improving a lot with better compiler features, new languages like Rust, and changes to software design and testing. Containers are being used more but we still aren’t getting all the benefits of that. TPM has become usable in the last few years and we are only starting to take advantage of what it can offer.

In 2012 BTRFS was still at an early stage of development and not many people wanted to use it in production, I was using it in production then and while I didn’t lose any data from bugs I did have some downtime because of BTRFS issues. Now BTRFS is quite solid for server use.

DDR4 was released in 2014 and gave significant improvements over DDR3 for performance and capacity. My home workstation now has 256G of DDR4 which wasn’t particularly expensive while the previous biggest system I owned had 96G of DDR3 RAM. Now DDR5 is available to again increase performance and size while also making DDR4 cheap on the second hand market.

This isn’t a comprehensive list of all advances in the computer industry over the last 12 years or so, it’s just some things that seem particularly noteworthy to me.

Please comment about what you think are the most noteworthy advances I didn’t mention.

10 July, 2024 06:55AM by etbe

hackergotchi for Freexian Collaborators

Freexian Collaborators

Debian Contributions: YubiHSM packaging, unschroot, live-patching, and more! (by Stefano Rivera)

Debian Contributions: 2024-06

Contributing to Debian is part of Freexian’s mission. This article covers the latest achievements of Freexian and their collaborators. All of this is made possible by organizations subscribing to our Long Term Support contracts and consulting services.

YubiHSM packaging, by Colin Watson

Freexian is starting to use YubiHSM devices (hardware security modules) as part of some projects, and we wanted to have the supporting software directly in Debian rather than needing to use third-party repositories. Since Yubico publish everything we need under free software licences, Colin packaged yubihsm-connector, yubihsm-shell, and python-yubihsm from https://developers.yubico.com/, in some cases based partly on the upstream packaging, and got them all into Debian unstable. Backports to bookworm will be forthcoming once they’ve all reached testing.

unschroot by Helmut Grohne

Following an in-person discussion at MiniDebConf Berlin, Helmut attempted splitting the containment functionality of sbuild --chroot-mode=unshare into a dedicated tool interfacing with sbuild as a variant of --chroot-mode=schroot providing a sufficiently compatible interface.

While this seemed technically promising initially, a discussion on debian-devel indicated a desire to rely on an existing container runtime such as podman instead of using another Debian-specific tool with unclear long term maintenance. None of the existing container runtimes meet the specific needs of sbuild, so further advancing this matter implies a compromise one way or another.

Linux live-patching, by Santiago Ruano Rincón

In collaboration with Emmanuel Arias, Santiago is working on the development of linux live-patching for Debian. For the moment, this is in an exploratory phase, that includes how to handle the different patches that will need to be provided. kpatch could help significantly in this regard. However, kpatch was removed from unstable because there are some RC bugs affecting the version that was present in Debian unstable. Santiago packaged the most recent upstream version (0.9.9) and filed an Intent to Salvage bug. Santiago is waiting for an ACK by the maintainer, and will upload to unstable after July 10th, following the package salvaging rules. While kpatch 0.9.9 fixes the main issues, it still needs some work to properly support Debian and the Linux kernel versions packaged in our distribution. More on this in the report next month.

Salsa CI, by Santiago Ruano Rincón

The work by Santiago in Salsa CI this month includes a merge request to ease testing how the production images are built from the changes introduced by future merge requests. By default, the pipelines triggered by a merge request build a subset of the images built for production, to reduce the use of resources, and because most of the time the subset of staging images is enough to test the proposed modifications. However, sometimes it is needed to test how the full set of production images is built, and the above mentioned MR helps to do that. The changes include documentation, so hopefully this will make it easier to test future contributions.

Also, for being able to include support for RISC-V, Salsa CI needs to replace kaniko as the tool used to build the images. Santiago tested buildah, but there are some issues when pushing built images for non-default platform architectures (i386, armhf, armel) to the container registry. Santiago will continue to work on this to find a solution.

Miscellaneous contributions

  • Stefano Rivera prepared updates for a number of Python modules.
  • Stefano uploaded the latest point release of Python 3.12 and the latest Python 3.13 beta. Both uncovered upstream regressions that had to be addressed.
  • Stefano worked on preparations for DebConf 24.
  • Stefano helped SPI to reconcile their financial records for DebConf 23.
  • Colin did his usual routine work on the Python team, upgrading 36 packages to new upstream versions (including fixes for four CVEs in python-aiohttp), fixing RC bugs in ipykernel, ipywidgets, khard, and python-repoze.sphinx.autointerface, and packaging zope.deferredimport which was needed for a new upstream version of python-persistent.
  • Colin removed the user_readenv option from OpenSSH’s PAM configuration (#1018260), and prepared a release note.
  • Thorsten Alteholz uploaded a new upstream version of cups.
  • Nicholas Skaggs updated xmacro to support reproducible builds (#1014428), DEP-3 and DEP-5 compatibility, along with utilizing hardening build flags. Helmut supported and uploaded package.
  • As a result of login having become non-essential, Helmut uploaded debvm to unstable and stable and fixed a crossqa.debian.net worker.
  • Santiago worked on the Content Team activities for DebConf24. Together with other DebConf25 team members, Santiago wrote a document for the head of the venue to describe the project of the conference.

10 July, 2024 12:00AM by Stefano Rivera

July 09, 2024

Simon Josefsson

Towards Idempotent Rebuilds?

After rebuilding all added/modified packages in Trisquel, I have been circling around the elephant in the room: 99% of the binary packages in Trisquel comes from Ubuntu, which to a large extent are built from Debian source packages. Is it possible to rebuild the official binary packages identically? Does anyone make an effort to do so? Does anyone care about going through the differences between the official package and a rebuilt version? Reproducible-build.org‘s effort to track reproducibility bugs in Debian (and other systems) is amazing. However as far as I know, they do not confirm or deny that their rebuilds match the official packages. In fact, typically their rebuilds do not match the official packages, even when they say the package is reproducible, which had me surprised at first. To understand why that happens, compare the buildinfo file for the official coreutils 9.1-1 from Debian bookworm with the buildinfo file for reproducible-build.org’s build and you will see that the SHA256 checksum does not match, but still they declare it as a reproducible package. As far as I can tell of the situation, the purpose of their rebuilds are not to say anything about the official binary build, instead the purpose is to offer a QA service to maintainers by performing two builds of a package and declaring success if both builds match.

I have felt that something is lacking, and months have passed and I haven’t found any project that address the problem I am interested in. During my earlier work I created a project called debdistreproduce which performs rebuilds of the difference between two distributions in a GitLab pipeline, and display diffoscope output for further analysis. A couple of days ago I had the idea of rewriting it to perform rebuilds of a single distribution. A new project debdistrebuild was born and today I’m happy to bless it as version 1.0 and to announces the project! Debdistrebuild has rebuilt the top-50 popcon packages from Debian bullseye, bookworm and trixie, on amd64 and arm64, as well as Ubuntu jammy and noble on amd64, see the summary status page for links. This is intended as a proof of concept, to allow people experiment with the concept of doing GitLab-based package rebuilds and analysis. Compare how Guix has the guix challenge command.

Or I should say debdistrebuild has attempted to rebuild those distributions. The number of identically built packages are fairly low, so I didn’t want to waste resources building the rest of the archive until I understand if the differences are due to consequences of my build environment (plain apt-get build-dep followed by dpkg-buildpackage in a fresh container), or due to some real difference. Summarizing the results, debdistrebuild is able to rebuild 34% of Debian bullseye on amd64, 36% of bookworm on amd64, 32% of bookworm on arm64. The results for trixie and Ubuntu are disappointing, below 10%.

So what causes my rebuilds to be different from the official rebuilds? Some are trivial like the classical problem of varying build paths, resulting in a different NT_GNU_BUILD_ID causing a mismatch. Some are a bit strange, like a subtle difference in one of perl’s headers file. Some are due to embedded version numbers from a build dependency. Several of the build logs and diffoscope outputs doesn’t make sense, likely due to bugs in my build scripts, especially for Ubuntu which appears to strip translations and do other build variations that I don’t do. In general, the classes of reproducibility problems are the expected. Some are assembler differences for GnuPG’s gpgv-static, likely triggered by upload of a new version of gcc after the original package was built. There are at least two ways to resolve that problem: either use the same version of build dependencies that were used to produce the original build, or demand that all packages that are affected by a change in another package are rebuilt centrally until there are no more differences.

The current design of debdistrebuild uses the latest version of a build dependency that is available in the distribution. We call this a “idempotent rebuild“. This is usually not how the binary packages were built originally, they are often built against earlier versions of their build dependency. That is the situation for most binary distributions.

Instead of using the latest build dependency version, higher reproducability may be achieved by rebuilding using the same version of the build dependencies that were used during the original build. This requires parsing buildinfo files to find the right version of the build dependency to install. We believe doing so will lead to a higher number of reproducibly built packages. However it begs the question: can we rebuild that earlier version of the build dependency? This circles back to really old versions and bootstrappable builds eventually.

While rebuilding old versions would be interesting on its own, we believe that is less helpful for trusting the latest version and improving a binary distribution: it is challenging to publish a new version of some old package that would fix a reproducibility bug in another package when used as a build dependency, and then rebuild the later packages with the modified earlier version. Those earlier packages were already published, and are part of history. It may be that ultimately it will no longer be possible to rebuild some package, because proper source code is missing (for packages using build dependencies that were never part of a release); hardware to build a package could be missing; or that the source code is no longer publicly distributable.

I argue that getting to 100% idempotent rebuilds is an interesting goal on its own, and to reach it we need to start measure idempotent rebuild status.

One could conceivable imagine a way to rebuild modified versions of earlier packages, and then rebuild later packages using the modified earlier packages as build dependencies, for the purpose of achieving higher level of reproducible rebuilds of the last version, and to reach for bootstrappability. However, it may be still be that this is insufficient to achieve idempotent rebuilds of the last versions. Idempotent rebuilds are different from a reproducible build (where we try to reproduce the build using the same inputs), and also to bootstrappable builds (in which all binaries are ultimately built from source code). Consider a cycle where package X influence the content of package Y, which in turn influence the content of package X. These cycles may involve several packages, and it is conceivable that a cycle could be circular and infinite. It may be difficult to identify these chains, and even more difficult to break them up, but this effort help identify where to start looking for them. Rebuilding packages using the same build dependency versions as were used during the original build, or rebuilding packages using a bootsrappable build process, both seem orthogonal to the idempotent rebuild problem.

Our notion of rebuildability appears thus to be complementary to reproducible-builds.org’s definition and bootstrappable.org’s definition. Each to their own devices, and Happy Hacking!

Addendum about terminology: With “idempotent rebuild” I am talking about a rebuild of the entire operating system, applied to itself. Compare how you build the latest version of the GNU C Compiler: it first builds itself using whatever system compiler is available (often an earlier version of gcc) which we call step 1. Then step 2 is to build a copy of itself using the compiler built in step 1. The final step 3 is to build another copy of itself using the compiler from step 2. Debian, Ubuntu etc are at step 1 in this process right now. The output of step 2 and step 3 ought to be bit-by-bit identical, or something is wrong. The comparison between step 2 and 3 is what I refer to with an idempotent rebuild. Of course, most packages aren’t a compiler that can compile itself. However entire operating systems such as Trisquel, PureOS, Ubuntu or Debian are (hopefully) a self-contained system that ought to be able to rebuild itself to an identical copy. Or something is amiss. The reproducible build and bootstrappable build projects are about improve the quality of step 1. The property I am interested is the identical rebuild and comparison in step 2 and 3. I feel the word “idempotent” describes the property I’m interested in well, but I realize there may be better ways to describe this. Ideas welcome!

09 July, 2024 10:16PM by simon

July 08, 2024

Thorsten Alteholz

My Debian Activities in June 2024

FTP master

This month I accepted 270 and rejected 23 packages. The overall number of packages that got accepted was 279.

Debian LTS

This was my hundred-twentieth month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.

During my allocated time I uploaded or worked on:

  • [DLA 3826-1] cups security update for one CVE to prevent arbitrary chmod of files
  • [#1073519] bullseye-pu: cups/2.3.3op2-3+deb11u7 to fix one CVE
  • [#1073518] bookworm-pu: cups/2.4.2-3+deb12u6 to fix one CVE
  • [#1073519] bullseye-pu: cups/2.3.3op2-3+deb11u7 package upload
  • [#1073518] bookworm-pu: cups/2.4.2-3+deb12u6 package upload
  • [#1074438] bullseye-pu: cups/2.3.3op2-3+deb11u8 to fix an upstream regression of the last upload
  • [#1074439] bookworm-pu: cups/2.4.2-3+deb12u7 to fix an upstream regression of the last upload
  • [#1074438] bullseye-pu: cups/2.3.3op2-3+deb11u8 package upload
  • [#1074439] bookworm-pu: cups/2.4.2-3+deb12u7 package upload
  • [#1055802] bookworm-pu: package qtbase-opensource-src/5.15.8+dfsg-11+deb12u1 package upload

This month handling of the CVE of cups was a bit messy. After lifting the embargo of the CVE, a published patch did not work with all possible combinations of the configuration. In other words, in cases of having only one local domain socket configured, the cupsd did not start and failed with a strange error. Anyway, upstream published a new set of patches, which made cups work again. Unfortunately this happended just before the latest point release for Bullseye and Bookworm, so that the new packages did not make it into the release, but stopped in the corresponding p-u-queues: stable-p-u and old-p-u.

I also continued to work on tiff and last but not least did a week of FD and attended the monthly LTS/ELTS meeting.

Debian ELTS

This month was the seventy-first ELTS month. During my allocated time I tried to upload a new version of cups for Jessie and Stretch. Unfortunately this was stopped due to an autopkgtest error, which I could not reproduce yet.

I also wanted to finally upload a fixed version of exim4. Unfortunately this was stopped due to lots of CI-jobs for Buster. Updates for Buster are now also availble from ELTS, so some stuff had to prepared before the actual switch end of June. Additionally everything was delayed due to a crash of the CI worker. All in all this month was rather ill-fated. At least the exim4 upload will happen/already happened in July.

I also continued to work on an update for libvirt, did a week of FD and attended the LTS/ELTS meeting.

Debian Printing

This month I uploaded new upstream or bugfix versions of:

This work is generously funded by Freexian!

Debian Astro

This month I uploaded a new upstream or bugfix version of:

All of those uploads are somehow related to /usr-move.

Debian IoT

This month I uploaded new upstream or bugfix versions of:

Debian Mobcom

The following packages have been prepared by the GSoC student Nathan:

misc

This month I uploaded new upstream or bugfix versions of:

Here as well all uploads are somehow related to /usr-move

08 July, 2024 06:27PM by alteholz

hackergotchi for Dirk Eddelbuettel

Dirk Eddelbuettel

RcppArmadillo 14.0.0-1 on CRAN: New Upstream

armadillo image

Armadillo is a powerful and expressive C++ template library for linear algebra and scientific computing. It aims towards a good balance between speed and ease of use, has a syntax deliberately close to Matlab, and is useful for algorithm development directly in C++, or quick conversion of research code into production environments. RcppArmadillo integrates this library with the R environment and language–and is widely used by (currently) 1158 other packages on CRAN, downloaded 35.1 million times (per the partial logs from the cloud mirrors of CRAN), and the CSDA paper (preprint / vignette) by Conrad and myself has been cited 587 times according to Google Scholar.

Conrad released a new major upstream version 14.0.0 a couple of days ago. We had been testing this new version extensively over several rounds of reverse-dependency checks across all 1100+ packages. This revealed nine packages requiring truly minor adjustments—which eight maintainers made in a matter of days; all this was coordinated in issue #443. Following the upload, CRAN noticed one more issue (see issue #446) but this turned out to be local to the package. There are also renewed deprecation warnings with some Armadillo changes which we will need to address one-by-one. Last but not least with this release we also changed the package versioning scheme to follow upstream Armadillo more closely.

The set of changes since the last CRAN release follows.

Changes in RcppArmadillo version 14.0.0-1 (2024-07-05)

  • Upgraded to Armadillo release 14.0.0 (Stochastic Parrot)

    • C++14 is now the minimum recommended C++ standard

    • Faster handling of compound expressions by as_scalar(), accu(), dot()

    • Faster interactions between sparse and dense matrices

    • Expanded stddev() to handle sparse matrices

    • Expanded relational operators to handle expressions between sparse matrices and scalars

    • Added .as_dense() to obtain dense vector/matrix representation of any sparse matrix expression

    • Updated physical constants to NIST 2022 CODATA values

  • New package version numbering scheme following upstream versions

  • Re-enabling ARMA_IGNORE_DEPRECATED_MARKE for silent CRAN builds

Courtesy of my CRANberries, there is a diffstat report relative to previous release. More detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the Rcpp R-Forge page.

If you like this or other open-source work I do, you can sponsor me at GitHub.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

08 July, 2024 06:39AM

July 07, 2024

Niels Thykier

Improving packaging file detection in Debian

Debian packaging consists of a directory (debian/) containing a number of "hard-coded" filenames such as debian/control, debian/changelog, debian/copyright. In addition to these, many packages will also use a number of optional files that are named via a pattern such as debian/{{PACKAGE}}.install.

At a high level the patterns looks deceptively simple. However, if you start working on trying to automatically classify files in debian/ (which could be helpful to tell the user they have a typo in the filename), you will quickly realize these patterns are not machine friendly at all for this purpose.

The patterns deconstructed

To appreciate the problem fully, here is a primer on the pattern and all its issues. If you are already well-versed in these, you might want to skip the section.

The most common patterns are debian/package.stem or debian/stem and usually the go to example is the install stem ( a concrete example being debian/debhelper.install). However, the full pattern consists of 4 parts where 3 of them are optional.

  • The package name followed by a period. Optional, but must be the first if present.
  • The name segment followed by a period. Optional, but must appear between the package name (if present) and the stem. If the package name is not present, then the name segment must be first.
  • The stem. Mandatory.
  • An architecture restriction prefixed by a period. Optional, must appear after the stem if present.

To visualize it with [foo] to mark optional parts, it looks like debian/[PACKAGE.][NAME.]STEM[.ARCH]

Detecting whether a given file is in fact a packaging file now boils down to reverse engineering its name against this pattern. Again, so far, it might still look manageable. One major complication is that every part (except ARCH) can contain periods. So a trivial "split by period" is not going to cut it. As an example:

debian/g++-3.0.user.service

This example is deliberately crafted to be ambiguous and show this problem in its full glory. This file name can be in multiple ways:

  • Is the stem service or user.service? (both are known stems from dh_installsystemd and dh_installsystemduser respectively). In fact, it can be both at the same time with "clever" usage of --name=user passed to dh_installsystemd.
  • The g++-3.0 can be a package prefix or part of the name segment. Even if there is a g++-3.0 package in debian/control, then debhelper (until compat 15) will still happily match this file for the main package if you pass --name=g++-3.0 to the helper. Side bar: Woe is you if there is a g++-3 and a g++-3.0 package in debian/control, then we have multiple options for the package prefix! Though, I do not think that happens in practice.

Therefore, there are a lot of possible ways to split this filename that all matches the pattern but with vastly different meaning and consequences.

Making detection practical

To make this detection practical, lets look at the first problems that we need to solve.

  1. We need the possible stems up front to have a chance at all. When multiple stems are an option, go for the longest match (that is, the one with most periods) since --name is rare and "code golfing" is even rarer.
  2. We can make the package prefix mandatory for files with the name segment. This way, the moment there is something before the stem, we know the package prefix will be part of it and can cut it. It does not solve the ambiguity if one package name is a prefix of another package name (from the same source), but it still a lot better. This made its way into debhelper compat 15 and now it is "just" a slow long way to a better future.

A simple solution to the first problem could be to have a static list of known stems. That will get you started but the debhelper eco-system strive on decentralization, so this feels like a mismatch.

There is also a second problem with the static list. Namely, a given stem is only "valid" if the command in question is actually in use. Which means you now need to dumpster dive into the mess that is Turning-complete debhelper configuration file known as debian/rules to fully solve that. Thanks to the Turning-completeness, we will never get a perfect solution for a static analysis.

Instead, it is time to back out and instead apply some simplifications. Here is a sample flow:

  1. Check whether the dh sequencer is used. If so, use some heuristics to figure out which addons are used.
  2. Delegate to dh_assistant to figure out which commands will be used and which debhelper config file stems it knows about. Here we need to know which sequences are in use from step one (if relevant). Combine this with any other sources for stems you have.
  3. Deconstruct all files in debian/ against the stems and known package names from debian/control. In theory, dumpster diving after --name options would be helpful here, but personally I skipped that part as I want to keep my debian/rules parsing to an absolute minimum.

With this logic, you can now:

  • Provide typo detection of the stem (debian/foo.intsall -> debian/foo.install) provided to have adequate handling of the corner cases (such as debian/*.conf not needing correction into debian/*.config)
  • Detect possible invalid package prefix (debian/foo.install without foo being a package). Note this has to be a weak warning unless the package is using debhelper compat 15 or you dumpster dived to validate that dh_install was not passed dh_install --name foo. Agreed, no one should do that, but they can and false positives are the worst kind of positives for a linting tool.
  • With some limitations, detect files used without the relevant command being active. As an example, the some integration modes of debputy removes dh_install, so a debian/foo.install would not be used.
  • Associate a given file with a given command to assist users with the documentation look up. Like debian/foo.user.service is related to dh_installsystemduser, so man dh_installsystemduser is a natural start for documentation.

I have added the logic for all these features in debputy though the documentation association is currently not in a user facing command. All the others are now diagnostics emitted by debputy in its editor support mode (debputy lsp server) or via debputy lint. In the editor mode, the diagnostics are currently associated with the package name in debian/control due to technical limitations of how the editor integration works.

Some of these features will the latest version of debhelper (moving target at times). Check with debputy lsp features for the Extra dh support feature, which will be enabled if you got all you need.

Note: The detection is currently (mostly) ignoring files with architecture restrictions. That might be lifted in the future. However, architecture restricted config files tend to be rare, so they were not a priority at this point. Additionally, debputy for technical reasons ignores stem typos with multiple matches. That sadly means that typos of debian/docs will often be unreported due to its proximity to debian/dirs and vice versa.

Diving a bit deeper on getting the stems

To get the stems, debputy has 3 primary sources:

  1. Its own plugins can provide packager provided files. These are only relevant if the package is using debputy.
  2. It is als possible to provide a debputy plugin that identifies packaging files (either static or named ones). Though in practice, we probably do not want people to roll their own debputy plugin for this purpose, since the detection only works if the plugin is installed. I have used this mechanism to have debhelper provide a debhelper-documentation plugin to enrich the auto-detected data and we can assume most people interested in this feature would have debhelper installed.
  3. It asks dh_assistant list-guessed-dh-config-files for config files, which is covered below.

The dh_assistant command uses the same logic as dh to identify the active add-ons and loads them. From there, it scans all commands mentioned in the sequence for the PROMISE: DH NOOP WITHOUT ...-hint and a new INTROSPECTABLE: CONFIG-FILES ...-hint. When these hints reference a packaging file (as an example, via pkgfile(foo)) then dh_assistant records that as a known packaging file for that helper.

Additionally, debhelper now also tracks commands that were removed from the sequence. Several of the dh_assistant subcommand now use this to enrich their (JSON) output with notes about these commands being known but not active.

The end result

With all of this work, you now get:

$ apt satisfy 'dh-debputy (>= 0.1.43~), debhelper (>= 13.16~), python3-lsprotocol, python3-levenshtein'
# For demo purposes, pull two known repos (feel free to use your own packages here)
$ git clone https://salsa.debian.org/debian/debhelper.git -b debian/13.16
$ git clone https://salsa.debian.org/debian/debputy.git -b debian/0.1.43
$ cd debhelper
$ mv debian/debhelper.install debian/debhelper.intsall
$ debputy lint
warning: File: debian/debhelper.intsall:1:0:1:0: The file "debian/debhelper.intsall" is likely a typo of "debian/debhelper.install"
    File-level diagnostic
$ mv debian/debhelper.intsall debian/debhleper.install
$ debputy lint
warning: File: debian/debhleper.install:1:0:1:0: Possible typo in "debian/debhleper.install". Consider renaming the file to "debian/debhelper.debhleper.install" or "debian/debhelper.install" if it is intended for debhelper
    File-level diagnostic
$ cd ../debputy
$ touch debian/install
$ debputy lint --no-warn-about-check-manifest
warning: File: debian/install:1:0:1:0: The file debian/install is related to a command that is not active in the dh sequence with the current addons
    File-level diagnostic

As mentioned, you also get these diagnostics in the editor via the debputy lsp server feature. Here the diagnostics appear in debian/control over the package name for technical reasons.

The editor side still needs a bit more work. Notably, changes to the filename is not triggered automatically and will first be caught on the next change to debian/control. Likewise, changes to debian/rules to add --with to dh might also have some limitations depending on the editor. Saving both files and then triggering an edit of debian/control seems to work reliable but ideally it should not be that involved.

The debhelper side could also do with some work to remove the unnecessary support for the name segment with many file stems that do not need them and announce that to debputy.

Anyhow, it is still a vast improvement over the status quo that was "Why is my file silently ignored!?".

07 July, 2024 12:00PM by Niels Thykier

July 05, 2024

Sahil Dhiman

Atleast Not Written by an AI

I keep on going back and correcting bootload of grammatical and other errors in my posts here. I somewhat feel embarrassed how such mistakes slip through when I was proofreading. Back then it was all good and suddenly this mistake cropped up in my text, which everyone might have already noticed by now. A thought just stuck around that. Those mistakes signify that the text is written by a real human, and humans makes mistakes. :)

PS - Even LanguageTool (non-premium) couldn’t identify those errors.

05 July, 2024 04:19PM

Reproducible Builds (diffoscope)

diffoscope 272 released

The diffoscope maintainers are pleased to announce the release of diffoscope version 272. This version includes the following changes:

[ Chris Lamb]
* Move away from using DSA OpenSSH keys in tests; support has been removed
  in OpenSSH 9.8p1. (Closes: reproducible-builds/diffoscope#382)
* Move to assert_diff helper in test_openssh_pub_key.py
* Update copyright years.

You find out more by visiting the project homepage.

05 July, 2024 12:00AM

July 04, 2024

Arturo Borrero González

Wikimedia Toolforge: migrating Kubernetes from PodSecurityPolicy to Kyverno

Le château de Valère et le Haut de Cry en juillet 2022 Christian David, CC BY-SA 4.0, via Wikimedia Commons

This post was originally published in the Wikimedia Tech blog, authored by Arturo Borrero Gonzalez.

Summary: this article shares the experience and learnings of migrating away from Kubernetes PodSecurityPolicy into Kyverno in the Wikimedia Toolforge platform.

Wikimedia Toolforge is a Platform-as-a-Service, built with Kubernetes, and maintained by the Wikimedia Cloud Services team (WMCS). It is completely free and open, and we welcome anyone to use it to build and host tools (bots, webservices, scheduled jobs, etc) in support of Wikimedia projects.

We provide a set of platform-specific services, command line interfaces, and shortcuts to help in the task of setting up webservices, jobs, and stuff like building container images, or using databases. Using these interfaces makes the underlying Kubernetes system pretty much invisible to users. We also allow direct access to the Kubernetes API, and some advanced users do directly interact with it.

Each account has a Kubernetes namespace where they can freely deploy their workloads. We have a number of controls in place to ensure performance, stability, and fairness of the system, including quotas, RBAC permissions, and up until recently PodSecurityPolicies (PSP). At the time of this writing, we had around 3.500 Toolforge tool accounts in the system. We early adopted PSP in 2019 as a way to make sure Pods had the correct runtime configuration. We needed Pods to stay within the safe boundaries of a set of pre-defined parameters. Back when we adopted PSP there was already the option to use 3rd party agents, like OpenPolicyAgent Gatekeeper, but we decided not to invest in them, and went with a native, built-in mechanism instead.

In 2021 it was announced that the PSP mechanism would be deprecated, and removed in Kubernetes 1.25. Even though we had been warned years in advance, we did not prioritize the migration of PSP until we were in Kubernetes 1.24, and blocked, unable to upgrade forward without taking actions.

The WMCS team explored different alternatives for this migration, but eventually we decided to go with Kyverno as a replacement for PSP. And so with that decision it began the journey described in this blog post.

First, we needed a source code refactor for one of the key components of our Toolforge Kubernetes: maintain-kubeusers. This custom piece of software that we built in-house, contains the logic to fetch accounts from LDAP and do the necessary instrumentation on Kubernetes to accommodate each one: create namespace, RBAC, quota, a kubeconfig file, etc. With the refactor, we introduced a proper reconciliation loop, in a way that the software would have a notion of what needs to be done for each account, what would be missing, what to delete, upgrade, and so on. This would allow us to easily deploy new resources for each account, or iterate on their definitions.

The initial version of the refactor had a number of problems, though. For one, the new version of maintain-kubeusers was doing more filesystem interaction than the previous version, resulting in a slow reconciliation loop over all the accounts. We used NFS as the underlying storage system for Toolforge, and it could be very slow because of reasons beyond this blog post. This was corrected in the next few days after the initial refactor rollout. A side note with an implementation detail: we stored a configmap on each account namespace with the state of each resource. Storing more state on this configmap was our solution to avoid additional NFS latency.

I initially estimated this refactor would take me a week to complete, but unfortunately it took me around three weeks instead. Previous to the refactor, there were several manual steps and cleanups required to be done when updating the definition of a resource. The process is now automated, more robust, performant, efficient and clean. So in my opinion it was worth it, even if it took more time than expected.

Then, we worked on the Kyverno policies themselves. Because we had a very particular PSP setting, in order to ease the transition, we tried to replicate their semantics on a 1:1 basis as much as possible. This involved things like transparent mutation of Pod resources, then validation. Additionally, we had one different PSP definition for each account, so we decided to create one different Kyverno namespaced policy resource for each account namespace — remember, we had 3.5k accounts.

We created a Kyverno policy template that we would then render and inject for each account.

For developing and testing all this, maintain-kubeusers and the Kyverno bits, we had a project called lima-kilo, which was a local Kubernetes setup replicating production Toolforge. This was used by each engineer in their laptop as a common development environment.

We had planned the migration from PSP to Kyverno policies in stages, like this:

  1. update our internal template generators to make Pod security settings explicit
  2. introduce Kyverno policies in Audit mode
  3. see how the cluster would behave with them, and if we had any offending resources reported by the new policies, and correct them
  4. modify Kyverno policies and set them in Enforce mode
  5. drop PSP

In stage 1, we updated things like the toolforge-jobs-framework and tools-webservice.

In stage 2, when we deployed the 3.5k Kyverno policy resources, our production cluster died almost immediately. Surprise. All the monitoring went red, the Kubernetes apiserver became irresponsibe, and we were unable to perform any administrative actions in the Kubernetes control plane, or even the underlying virtual machines. All Toolforge users were impacted. This was a full scale outage that required the energy of the whole WMCS team to recover from. We temporarily disabled Kyverno until we could learn what had occurred.

This incident happened despite having tested before in lima-kilo and in another pre-production cluster we had, called Toolsbeta. But we had not tested that many policy resources. Clearly, this was something scale-related. After the incident, I went on and created 3.5k Kyverno policy resources on lima-kilo, and indeed I was able to reproduce the outage. We took a number of measures, corrected a few errors in our infrastructure, reached out to the Kyverno upstream developers, asking for advice, and at the end we did the following to accommodate the setup to our needs:

  • corrected the external HAproxy kubernetes apiserver health checks, from checking just for open TCP ports, to actually checking the /healthz HTTP endpoint, which more accurately reflected the health of each k8s apiserver.
  • having a more realistic development environment. In lima-kilo, we created a couple of helper scripts to create/delete 4000 policy resources, each on a different namespace.
  • greatly over-provisioned memory in the Kubernetes control plane servers. This is, bigger memory in the base virtual machine hosting the control plane. Scaling the memory headroom of the apiserver would prevent it from running out of memory, and therefore crashing the whole system. We went from 8GB RAM per virtual machine to 32GB. In our cluster, a single apiserver pod could eat 7GB of memory on a normal day, so having 8GB on the base virtual machine was clearly not enough. I also sent a patch proposal to Kyverno upstream documentation suggesting they clarify the additional memory pressure on the apiserver.
  • corrected resource requests and limits of Kyverno, to more accurately describe our actual usage.
  • increased the number of replicas of the Kyverno admission controller to 7, so admission requests could be handled more timely by Kyverno.

I have to admit, I was briefly tempted to drop Kyverno, and even stop pursuing using an external policy agent entirely, and write our own custom admission controller out of concerns over performance of this architecture. However, after applying all the measures listed above, the system became very stable, so we decided to move forward. The second attempt at deploying it all went through just fine. No outage this time 🙂

When we were in stage 4 we detected another bug. We had been following the Kubernetes upstream documentation for setting securityContext to the right values. In particular, we were enforcing the procMount to be set to the default value, which per the docs it was ‘DefaultProcMount’. However, that string is the name of the internal variable in the source code, whereas the actual default value is the string ‘Default’. This caused pods to be rightfully rejected by Kyverno while we figured the problem. I sent a patch upstream to fix this problem.

We finally had everything in place, reached stage 5, and we were able to disable PSP. We unloaded the PSP controller from the kubernetes apiserver, and deleted every individual PSP definition. Everything was very smooth in this last step of the migration.

This whole PSP project, including the maintain-kubeusers refactor, the outage, and all the different migration stages took roughly three months to complete.

For me there are a number of valuable reasons to learn from this project. For one, the scale is something to consider, and test, when evaluating a new architecture or software component. Not doing so can lead to service outages, or unexpectedly poor performances. This is in the first chapter of the SRE handbook, but we got a reminder the hard way 🙂

This post was originally published in the Wikimedia Tech blog, authored by Arturo Borrero Gonzalez.

04 July, 2024 09:00AM

hackergotchi for Samuel Henrique

Samuel Henrique

Debian's curl now supports HTTP3

tl;dr

Starting with curl 8.0.0-2, you can now use HTTP3.

curl --http3-only https://example.com

Or, if you would like to try it out in a container:

podman run debian:unstable apt install --update -y curl && curl --http3-only https://example.com

(in case you haven't noticed, apt now has the --update option for the upgrade and install commands, although not available on stable yet)

Availability
  • Debian unstable - Since 2024-07-02
  • Debian testing - Since 2024-07-18
  • Debian 12/bookworm backports - Expected by the end of August 2024.
  • Debian 12/bookworm - Due to the mechanisms we have in place to make sure Debian stable is in fact stable, we will never be able to ship this in the regular repository. Users can make use of the backports repositories instead.
  • Debian derivatives - Rolling releases will get it by the time it's on Debian testing (e.g.: Kali Linux). Stable derivatives only in their next major release.

The challenge

HTTP3 is fresh new, well... not really, but at least fresh enough that I'm not aware of any other Linux distribution supporting it on curl, the reason is likely two-fold:

  1. OpenSSL is not there yet

    OpenSSL still doesn't have proper HTTP3 support, and given that OpenSSL is so widely used, almost every curl distributor/packager will build curl with it and thus changing the TLS backend to something else is risky.

    Unfortunately, proper support for the OpenSSL libcurl is unlikely to come anytime before the end of this year, the OpenSSL performance is not good enough yet as of version 3.3.

    Daniel Stenberg has written about the state of this multiple times, most recently at HTTP/3 in curl mid 2024, if you're interested, I suggest reading through his other posts as well.

    Some might have noticed that nginx does support HTTP3 through OpenSSL, although when you look closely, it's not exactly perfect:

    An SSL library that provides QUIC support is recommended to build nginx, such as BoringSSL, LibreSSL, or QuicTLS. Otherwise, the OpenSSL compatibility layer will be used that does not support early data.

    As you can see, they don't recommend using OpenSSL, and when doing so, you don't get complete support.

  2. HTTP3 support for GnuTLS/nghttp3/ngtcp2 is recent

    The non-experimental support arrived back in October 2023, and so that's when I started seriously planning for this.

    curl has been working on HTTP3 support for years, and so it did support other TLS backends before that, but out of them, the one most feasible for a distribution to ship would be GnuTLS, which gets HTTP3 support through ngctp2 and nghttp3.

How it was done

The Debian curl package has historically shipped at least two variants of libcurl, an OpenSSL and a GnuTLS one.

The OpenSSL libcurl can't support HTTP3 for the reasons explained above, but the GnuTLS libcurl can (with ngtcp2 and nghtp3).

Debian packages can choose which version of libcurl to link against (without having to modify any upstream source code). Debian's "git" package being a famous example of a package that links against the GnuTLS libcurl.

Enabling HTTP3 on curl was done in three steps:

  1. Make sure all required dependencies fulfill the minimum requirements.
  2. Enable HTTP3 for GnuTLS libcurl.
  3. Change the libcurl used by the curl CLI, from OpenSSL to GnuTLS.

curl's HTTP3 support requires a somewhat recent version of nghttp3 and updating that required a transition (due to the SONAME bump), while we've also had months of freeze for transitions due to the time_t transition.

After the dependencies were in place, enabling HTTP3 for the GnuTLS libcurl was straightforward.

Then, for the last part, we had to switch the TLS backend used by the curl CLI. Doing the swap is also quite easy on the packaging level, but we have to consider the chances of this change breaking our users' environments.

Ensuring there are no breakages

The first thing to consider regarding breakages is that this change is not going to be pushed directly to the current Debian stable releases, it will be present in the next stable release (13/trixie) but the current one will stick to the version that's already shipped.

Secondly, we have to consider the risk of losing the ability to use certain parameters from the curl CLI which could be limited to the OpenSSL backend. During curl-up 2024, the curl developers pointed out the existence of a page that lists the TLS related options and the backends they work with.

Analysing that page, ignoring all of the options that are suffixed with "BLOB" (only pertinent to the library, not the CLI), the only one left which is attention worthy is CURLOPT_ECH.

This experimental feature requires a special build of OpenSSL, as ECH is not yet supported in OpenSSL releases. In contrast ECH is supported by the latest BoringSSL and wolfSSL releases.

As it turns out, Encrypted Client Hello is experimental and it's not supported by the vanilla OpenSSL.

This was enough of an investigation for me to go ahead with the change. Noting that even in the worst case scenario (we find a horrible regression), we can rollback without having affected a single stable release.

Now that the package is on Debian unstable, the CI tests (autopkgtest) of every package that depends on curl is currently running, the results are compared against the migration-reference (in this case, the curl CLI with OpenSSL, before the change).

If everything goes right, curl with HTTP3 support will migrate to Debian testing in around 5 days. If we spot any issues, we'll have to solve them first and it's going to be hard to predict how long it takes, although it's fair to expect less than a month.

Feedback

Feel free to join the Matrix room for the Debian curl maintainers:
https://matrix.to/#/#debian-curl-maintainers:matrix.org

Acknowledgements

It took us a bit longer than expected to be able to enable HTTP3, nonetheless it's still early enough to be excited about.

A lot of people were crucial to make this happen.

I should recognize in the first place, obviously, the curl developers and the developers of the supporting libraries: GnuTLS, nghttp3, ngtcp2. Participating in the curl-up 2024 conference helped me get motivated to push this through, besides becoming aware of the right documentation to research for impact.

On the Debian side, Sakirnth Nagarasa <sakirnth> was responsible for updating and taking care of the transition for nghttp3 and ngtcp2.

Also on the Debian side, I've got loads of help and support from the co-maintainers of the curl package: Sergio Durigan Junior <sergiodj> and Carlos Henrique Lima Melara <charles>.

Changes since publication

2024-07-18

  • Update date of availability for Debian testing and expected date for bookworm backports.
  • We have historically spoken Portuguese in the room but we'll switch to English in case anyone joins.

04 July, 2024 12:00AM by Unknown

July 03, 2024

hackergotchi for Mike Gabriel

Mike Gabriel

Polis - a FLOSS Tool for Civic Participation -- Initial Evaluation and Adaptation (episode 2/5)

Here comes the 2nd article of the 5-episode blog post series written by Guido Berhörster, member of staff at my company Fre(i)e Software GmbH.

Enjoy also this read on Guido's work on Polis,
Mike

Table of Contents of the Blog Post Series

  1. Introduction
  2. Initial evaluation and adaptation (this article)
  3. Issues extending Polis and adjusting our goals
  4. Creating (a) new frontend(s) for Polis
  5. Current status and roadmap

Polis - Initial evaluation and adaptation

The Polis code base consists of a number of components, the administration and participation interfaces, a common web backend, and a statistics processing server. Both frontends and the backend are written in a mixture of JavaScript and TypeScript, only the statistics processing server is written in Clojure.

In case of self hosting the preferred method of deployment is via Docker containers using Docker Compose or any other orchestrator. The participation frontend for conversations can either be used as a standalone web page or be embedded via an iframe.

For our planned use case we initially defined the following goals:

  • custom branding and the integration into different content management systems (CMS)
  • better support for mobile devices
  • mandatory authentication and support for a broader range of authentication methods, including self-hosted solutions and DigiD
  • support for alternative email sending services
  • GDPR compliance

After a preliminary evaluation of our own and consulting with Policy Lab UK who were also evaluating and testing Polis and had already made a range of improvements related to self-hosting as well as bug fixes and modernization changes we decided to take their work as a base for our adaptations with the intent of submitting generally useful changes back to the Polis project.

Subsequently, a number of changes were implemented, including the removal of hardcoded domain names, the elimination of unnecessary cookies and third-party requests, support for an alternative email sending service, and the option of disabling Facebook and X integration.

For the branding our approach was to add an option allowing websites which are embedding conversations in an iframe to load an alternative stylesheet for overriding the native Polis branding. For this to be practical we intended to use CSS custom properties for defining branding-related styles such as colors and fonts. That approach turned out to be problematic because although the Polis participation frontend stylesheet is generated via SCSS and some of the colors are parameterized, however, they are not used consistently throughout the SCSS stylesheets, unfortunately. In addition the frontend templates contain a large amount of hardcoded style attributes. While we succeeded in implementing user-defined stylesheets, it took a disproportionate amount of development resources to parameterize all used colors and fonts via CSS custom properties aggravated by the fact that the SCSS and template files are huge and contain many unused rules and code.

03 July, 2024 07:56PM by sunweaver

Ian Jackson

derive-deftly is nearing 1.x - call for review/testing

derive-deftly, the template-based derive-macro facility for Rust, has been a great success.

It’s coming up to time to declare a stable 1.x version. If you’d like to try it out, and have final comments / observations, now is the time.

Introduction to derive-deftly

Have you ever wished that you could that could write a new derive macro without having to mess with procedural macros?

You can!

derive-deftly lets you write a #[derive] macro, using a template syntax which looks a lot like macro_rules!:

use derive_deftly::{define_derive_deftly, Deftly};

define_derive_deftly! {
    ListVariants:

    impl $ttype {
        fn list_variants() -> Vec<&'static str> {
            vec![ $( stringify!( $vname ) , ) ]
        }
    }
}

#[derive(Deftly)]
#[derive_deftly(ListVariants)]
enum Enum {
    UnitVariant,
    StructVariant { a: u8, b: u16 },
    TupleVariant(u8, u16),
}

assert_eq!(
    Enum::list_variants(),
    ["UnitVariant", "StructVariant", "TupleVariant"],
);

Status

derive-deftly has a wide range of features, which can be used to easily write sophisticated and reliable derive macros. We’ve been using it in Arti, the Tor Project’s reimplementation of Tor in Rust, and we’ve found it very useful.

There is comprehensive reference documentation, and more discursive User Guide for a more gentle introduction. Naturally, everything is fully tested.

History

derive-deftly started out as a Tor Hackweek project. It used to be called derive-adhoc. But we renamed it because we found that many of the most interesting use cases were really not very ad-hoc at all.

Over the past months we’ve been ticking off our “1.0 blocker” tickets. We’ve taken the opportunity to improve syntax, terminology, and semantics. We hope we have now made the last breaking changes.

Plans - call for review/testing

In the near future, we plan to declare version 1.0. After 1.x, we intend to make breaking changes very rarely.

So, right now, we’d like last-minute feedback. Are there any wrinkles that need to be sorted out? Please file tickets or MRs on our gitlab. Ideally, anything which might imply breaking changes would be submitted on or before the 13th of August.

In the medium to long term, we have many ideas for how to make derive-deftly even more convenient, and even more powerful. But we are going to proceed cautiously, because we don’t want to introduce bad syntax or bad features, which will require difficult decisions in the future about forward compatibility.



comment count unavailable comments

03 July, 2024 06:32PM

Sahil Dhiman

RTI to NPL Regarding Their NTP Infrastructure

I became interested in Network Time Protocol (NTP) last year after learning how fundamental this protocol is to the functioning of the global Internet. NTP helps synchronize clocks on devices over the Internet, which is essential for secure browsing, timestamping, keeping everyone in sync or just checking what time it is. Computers usually have a hardware real-time clock (RTC) but that deviates over time, so an occasional sync over NTP is required to keep the time accurate. Many network and IoT devices don’t have hardware RTC so have even more reliance on NTP.

Accurate time keeping starts with reference clocks like atomic clocks, GPS etc. Multiple government standard agencies host these reference clocks, which are regarded as Stratum 0. Stratum 1 servers are known as primary servers, and directly connect to Stratum 0 clocks for time. Stratum 1 servers then distribute time to Stratum 2 and further down the hierarchy. Computers typically connects to one or more Stratum 1/2/3… servers to get their time.

Someone has to host these public Stratum 1,2,3… NTP servers. That’s what NTP pool, a global effort by volunteers does. They provide NTP servers for the public to use. As of today, there are 4700+ servers in the pool which are free to use for anyone.

Now let’s come to the reason for writing this post. Indian Computer Emergency Response Team (CERT-In) in April 2022 released a set of cybersecurity directions which set the alarm bells ringing. Internet Society (and almost everyone else) wrote about it.

And then there was this specific section about NTP:

All service providers, intermediaries, data centres, body corporate and Government organisations shall connect to the Network Time Protocol (NTP) Server of National Informatics Centre (NIC) or National Physical Laboratory (NPL) or with NTP servers traceable to these NTP servers, for synchronisation of all their ICT systems clocks. Entities having ICT infrastructure spanning multiple geographies may also use accurate and standard time source other than NPL and NIC, however it is to be ensured that their time source shall not deviate from NPL and NIC.

CSIR-National Physical Laboratory (NPL) is the official timekeeper for India and hosts the only public Stratum 1 clock in India, according to NTP pool website. So I was naturally curious to know what kind of infrastructure they’re running for NTP. India has a Right to Information (RTI) Act which, like the Freedom of Information Act (FOIA) in the United States, gives citizens rights to request information from governmental entities, to which they have to respond in under 30 days. So last year, I filed two sets of RTI (one after the first reply came) inquiring about NPL’s public NTP server setup.

The first RTI had some generic questions: RTI 1

First RTI. Click to enlarge

This gave a vague idea about the setup, so I sat down and came with some specific questions in the next RTI.

RTI 2

Second RTI. Click to enlarge

Feel free to make your conclusions from it now. Bear in mind these were filled last year so things might have changed. Do let me know if you have more information about it.

Update (07/07/2024): Found an article from Medianama about Indian government time servers with information sourced through RTI.

03 July, 2024 04:15PM

hackergotchi for Samuel Henrique

Samuel Henrique

Announcing wcurl: a curl wrapper to download files

tl;dr

Whenever you need to download files through the terminal and don't feel like using wget:

wcurl example.com/filename.txt

Manpage:
https://manpages.debian.org/unstable/curl/wcurl.1.en.html

Availability (comes installed with the curl package):
  • Debian unstable - Since 2024-07-02
  • Debian testing - Since 2024-07-18
  • Debian 12/bookworm backports - Expected by the end of August 2024.
  • Debian 12/bookworm - Depends on whether Debian's release team will approve it, it could be available in the next point release.
  • Debian derivatives - Rolling releases will get it by the time it's on Debian testing (e.g.: Kali Linux). Stable derivatives only in their next major release.

If you don't want to wait for the package update to arrive, you can always copy the script and place it in your /usr/bin, the code is here:
https://github.com/Debian/wcurl/blob/main/wcurl
https://salsa.debian.org/debian/wcurl/-/blob/main/wcurl?ref_type=heads

Smoother CLI experience

Starting with curl version 8.8.0-2, the Debian's curl package now ships a wcurl executable.

wcurl is the solution for those who just need to download files without having to remember curl's parameters for things like automatically naming the files.

Some people, myself included, would fall back to using wget whenever there was a need to download a file. Sometimes even installing wget just for that usecase. After all, it's easier to remember "apt install wget" rather than "curl -L -O -C - ...".

wcurl consists of a simple shell script that provides sane defaults for the curl invocation, for when the use case is to just download files.

By default, wcurl will:

  • Encode whitespaces in URLs;
  • Download multiple URLs in parallel if the installed curl's version is >= 7.66.0;
  • Follow redirects;
  • Automatically choose a filename as output;
  • Avoid overwriting files if the installed curl's version is >= 7.83.0 (--no-clobber);
  • Perform retries;
  • Set the downloaded file timestamp to the value provided by the server, if available;
  • Default to the protocol used as https if the URL doesn't contain any;
  • Disable curl's URL globbing parser so {} and [] characters in URLs are not treated specially.

Example to download a single file:

wcurl example.com/filename.txt

If you ever need to set a custom flag, you can make use of the --curl-options wcurl option, anything set there will be passed to the curl invocation. Just beware that if you need to set any custom flags, it's likely you will be better served by calling curl directly. The --curl-options option is there to allow for some flexibility in unforeseen circumstances.

The need for wcurl

I've always felt a bit ashamed of not remembering curl's parameters for downloading a file and automatically naming it, having resorted to wget most of the times this was needed (even installing wget when it wasn't there, just for this). I've spoken to a few other experienced people I know and confirmed what could be obvious to others: a lot of people struggle with this.

Recently, the curl project released the results of 2024's curl survey, which also showed this is as a much needed feature, just look at some of the answers:

Q: Which curl command line option do you think needs improvement and how?

-O, I really want wget like functionality where I don't have to specify the name

Downloading a file (like wget) could be improved - with automatic naming of the file

downloading files - wget is much cleaner

I wish the default behaviour when GETting a binary was to drop it on disk. That's the only reason 'wget foo.tgz" is still ingrained in my muscle memory .

Maybe have a way to download without specifying something in -o (the only reason i used wget still)

--remote-time should be default

--remote-name-all could really use a short flag

Q: If you miss support for something, tell us what!

"Write the data to the file named in the URL (or in redirects if I'm feeling daring), and timestamp the file to the last-modified-date". This is the main reason I'm still using wget.

I can finally feel less bad about falling back to wget due to not remembering the parameters I want.

Idealization vs. reality

I don't believe curl will ever change its default behavior in such a way that would accommodate this need, as that would have a side-effect of breaking things which expect the current behavior (the blast radius is literally the solar system).

This means a new executable needs to be shipped side-by-side with curl, an opportunity to start fresh and work with a more focused use case (to download files).

Ideally, this new executable would be maintained by the curl project, make use of libcurl under-the-hood, and be available everywhere. Nobody wants to worry if their systems have the tool or not, it should always be there.

Given I'm just a Debian Developer, with not as much free time as I wish, I've decided to write a simple shell script wrapper calling the curl CLI under-the-hood.

wcurl will come installed with the curl package from now on, and I will check with the release team about shipping it on the current Debian stable as well. Shipping wcurl in other distros will be up to them (Debian-derivatives should pick it up automatically, though).

We've tried to make it easy for anyone to ship this by using the curl license, keeping the script POSIX-compliant, and shipping a manpage.

Maybe if there's enough interest across distributions, someone might sign up for implementing this in upstream curl and increase its reach. I would be happy with the curl project reusing the wcurl name when that happens. It's unlikely that wcurl would be shipped by curl upstream as it is, assuming they would prefer a solution that uses libcurl direclty (more similar to curl the CLI, to maintain).

In the worst case, wcurl becomes a Debian-specific tool that only a few people are aware of, in the best case, it becomes the new go-to CLI tool for simply downloading files. I would be happy if at least someone other than me finds it useful.

Naming is hard

When I started working on it, I was calling the new executable "curld" (stands for "curl download"), but then when discussing this in one of our weekly calls in the Debian Brasília community, it was mentioned that this could be confused for a daemon.

We then settled for the name "wcurl", suggested by Carlos Henrique Lima Melara <charles>. It doesn't really stand for anything, but it's very easy to remember.

You know... "it's that wget alternative for when you want to use curl instead" :)

Feedback

I'm hosting the code on Github and Debian's GitLab instance, feel free to open an issue to provide feedback.
https://salsa.debian.org/debian/wcurl
https://github.com/Debian/wcurl

We also have a Matrix room for the Debian curl maintainers:
https://matrix.to/#/#debian-curl-maintainers:matrix.org

Acknowledgments

The idea for wcurl came a few days before the curl-up conference 2024. I've been thinking a lot about developer productivity in the terminal lately, different tools and better defaults. Before curl-up, I was also thinking about packaging improvements for the curl package. I don't remember what exactly happened, but I likely had to download something and felt a bit ashamed of maintaining curl and not remembering the parameters to download files the way I wanted.

I first discussed this idea in the conference, where I asked the participants about it and there were no concerns raised, and some people said I should give it a go. Participating in curl-up was a really great experience and I'm thankful for the interactions I've had there.

On the Debian side, I've got reviews of the code and manpage by Sergio Durigan Junior <sergiodj>, Guilherme Puida Moreira <puida> and Carlos Henrique Lima Melara <charles>. Sergio ended up rewriting the tool to be POSIX-compliant (my version was written in bash), so he takes all the credit for the portability.

Changes since publication

2024-07-18

  • Update date of availability for Debian testing and expected date for bookworm backports.
  • Mention charles as the person who suggested "wcurl" as a name.
  • Update wcurl's -o/--opts options, it's now just --curl-options.
  • Remove mention of language spoken in the Matrix room, we are using English now.
  • Update list of features of wcurl.

03 July, 2024 12:00AM by Unknown

July 02, 2024

Dima Kogan

vnlog.slurp() with non-numerical data

For a while now I'd see an annoying problem when trying to analyze data. I would be trying to import into numpy an innocuous-looking data file like this:

#  image   x y z temperature
image1.png 1 2 5 34
image2.png 3 4 1 35

As usual, I would be using vnlog.slurp() (a thin wrapper around numpy.loadtxt()) to read this in, but that doesn't work: the image filenames aren't parseable as numerical values. Up until now I would work around this by using the suprocess module to fork off a vnl-filter -p !image and then slurp that, but it's a pain and slow and has other issues. I just solved this conclusively using the numpy structured dtypes. I can now do this:

dtype = np.dtype([ ('image',       'U16'),
                   ('x y z',       int, (3,)),
                   ('temperature', float), ])

arr = vnlog.slurp("data.vnl", dtype=dtype)

This will read the image filename, the xyz points and the temperature into different sub-arrays, with different types each. Accessing the result looks like this:

print(arr['image'])
---> array(['image1.png', 'image2.png'], dtype='<U16')

print(arr['x y z'])
---> array([[1, 2, 5],
            [3, 4, 1]])

print(arr['temperature'])
---> array([34., 35.])

Notes:

  • The given structured dtype defines both how to organize the data, and which data to extract. So it can be used to read in only a subset of the available columns. Here I could have omitted the temperature column, for instance
  • Sub-arrays are allowed. In the example I could say either
    dtype = np.dtype([ ('image',       'U16'),
                       ('x y z',       int, (3,)),
                       ('temperature', float), ])
    

    or

    dtype = np.dtype([ ('image',       'U16'),
                       ('x',           int),
                       ('y',           int),
                       ('z',           int),
                       ('temperature', float), ])
    

    The latter would read x, y, z into separate, individual arrays. Sometime we want this, sometimes not.

  • Nested structured dtypes are not allowed. Fields inside other fields are not supported, since it's not clear how to map that to a flat vnlog legend
  • If a structured dtype is given, slurp() returns the array only, since the field names are already available in the dtype

We still do not support records with any null values (-). This could probably be handled with the converters kwarg of numpy.loadtxt(), but that sounds slow. I'll look at that later.

This is available today in vnlog 1.38.

02 July, 2024 05:38PM by Dima Kogan

hackergotchi for Bits from Debian

Bits from Debian

Bits from the DPL

Dear Debian community,

Statement on Daniel Pocock

The Debian project has successfully taken action to secure its trademarks and interests worldwide, as detailed in our press statement. I would like to personally thank everyone in the community who was involved in this process. I would have loved for you all to have spent your volunteer time on more fruitful things.

Debian Boot team might need help

I think I've identified the issue that finally motivated me to contact our teams: for a long time, I have had the impression that Debian is driven by several "one-person teams" (to varying extents of individual influence and susceptibility to burnout). As DPL, I see it as my task to find ways to address this issue and provide support.

I received private responses from Debian Boot team members, which motivated me to kindly invite volunteers to some prominent and highly visible fields of work that you might find personally challenging. I recommend subscribing to the Debian Boot mailing list to see where you might be able to provide assistance.

/usrmerge

Helmut Grohne confirmed that the last remaining packages shipping aliased files inside the package set relevant to debootstrap were uploaded. Thanks a lot for Helmut and all contributors that helped to implement DEP17.

Contacting more teams

I'd like to repeat that I've registered a BoF for DebConf24 in Busan with the following description:

This BoF is an attempt to gather as much as possible teams inside Debian to exchange experiences, discuss workflows inside teams, share their ways to attract newcomers etc.

Each participant team should prepare a short description of their work and what team roles (“openings”) they have for new contributors. Even for delegated teams (membership is less fluid), it would be good to present the team, explain what it takes to be a team member, and what steps people usually go to end up being invited to participate. Some other teams can easily absorb contributions from salsa MRs, and at some point people get commit access. Anyway, the point is that we work on the idea that the pathway to become a team member becomes more clear from an outsider point-of-view.

I'm lagging a bit behind my team contacting schedule and will not manage to contact every team before DebConf. As a (short) summary, I can draw some positive conclusions about my efforts to reach out to teams. I was able to identify some issues that were new to me and which I am now working on. Examples include limitations in Salsa and Salsa CI. I consider both essential parts of our infrastructure and will support both teams in enhancing their services.

Some teams confirmed that they are basically using some common infrastructure (Salsa team space, mailing lists, IRC channels) but that the individual members of the team work on their own problems without sharing any common work. I have also not read about convincing strategies to attract newcomers to the team, as we have established, for instance, in the Debian Med team.

DebConf attendance

The amount of money needed to fly people to South Korea was higher than usual, so the DebConf bursary team had to make some difficult decisions about who could be reimbursed for travel expenses. I extended the budget for diversity and newcomers, which enabled us to invite some additional contributors. We hope that those who were not able to come this year can make it next year to Brest or to MiniDebConf Cambridge or Toulouse

tag2upload

On June 12, Sean Whitton requested comments on the debian-vote list regarding a General Resolution (GR) about tag2upload. The discussion began with technical details but unfortunately, as often happens in long threads, it drifted into abrasive language, prompting the community team to address the behavior of an opponent of the GR supporters. After 560 emails covering technical details, including a detailed security review by Russ Allbery, Sean finally proposed the GR on June 27, 2024 (two weeks after requesting comments).

Firstly, I would like to thank the drivers of this GR and acknowledge the technical work behind it, including the security review. I am positively convinced that Debian can benefit from modernizing its infrastructure, particularly through stronger integration of Git into packaging workflows.

Sam Hartman provided some historical context [1], [2], [3], [4], noting that this discussion originally took place five years ago with no results from several similarly lengthy threads. My favorite summary of the entire thread was given by Gregor Herrmann, which reflects the same gut feeling I have and highlights a structural problem within Debian that hinders technical changes. Addressing this issue is definitely a matter for the Debian Project Leader, and I will try to address it during my term.

At the time of writing these bits, a proposal from ftpmaster, which is being continuously discussed, might lead to a solution. I was also asked to extend the GR discussion periods which I will do in separate mail.

Talk: Debian GNU/Linux for Scientific Research

I was invited to have a talk in the Systems-Facing Track of University of British Columbia (who is sponsoring rack space for several Debian servers). I admit it felt a bit strange to me after working more than 20 years for establishing Debian in scientific environments to be invited to such a talk "because I'm DPL". 😉

Kind regards Andreas.

02 July, 2024 05:00PM by Andreas Tille

hackergotchi for Mike Gabriel

Mike Gabriel

Polis - a FLOSS Tool for Civic Participation -- Introduction (episode 1/5)

This is the first article of a 5-episode blog post series written by Guido Berhörster, member of staff at my company Fre(i)e Software GmbH. Thanks, Guido for being on the Polis project.

Enjoy the read on the work Guido has been doing over the past months,
Mike

A team lead by Raoul Kramer/BetaBreak is currently adapting Polis for evaluation and testing by several Dutch provincial governments and central government ministries. Guido Berhörster (author of this article) who is an employee at Fre(i)e Software GmbH has been involved in this project as the main software developer. This series of blog posts describes how and why Polis was initially modified and adapted, what issues the team ran into and how this ultimately lead them to start a new Open Source project called Particiapp for accelerating the development of alternative Polis frontends compatible to but independent from the upstream project.

Table of Contents of the Blog Post Series

  1. Introduction (this article)
  2. Initial evaluation and adaptation
  3. Issues extending Polis and adjusting our goals
  4. Creating (a) new frontend(s) for Polis
  5. Current status and roadmap

Polis - The Introduction

What is Polis?

Polis is a platform for participation which helps to gather, analyze and understand viewpoints of large groups of participants on complex issues. In practical terms participants take part in “conversations” on a predefined topic by voting on statements or submitting their own statements (referred to as “comments” in Polis) for others to vote on1.

Through statistical analysis including machine learning participants are sorted into groups based on similarities in voting behavior. In addition, group-informed and overall consensus statements are identified and presented to participants in real-time. This allows for participants to react to and refine statements and either individually or through a predefined process to come to an overall consensus.

Furthermore, the order in which statements are presented to participants is influenced by a complex weighting system based on a number of factors such as variance, recency, and frequency of skipping. This so called “comment routing” is intended to facilitate a meaningful contribution of participants without requiring them to vote on each of a potentially huge number of statements 2.

Polis open-ended nature sets it apart from online surveys using pre-defined questions and allows its users to gather a more accurate picture of the public opinion. In contrast to a discussion forum or comment section where participants directly reply to each other, it discourages unproductive behavior such as provocations or personal attacks by not presenting statements in chronological order in combination with voting. Finally, its “comment routing” is intended to provide scalability towards a large number of participants which generate a potentially large number of statements.

The project was developed and is maintained by The Computational Democracy Project, a USA-based non-profit organization which provides a hosted version and offers related services. It is also released as Open Source software under the AGPL 3.0 license.

Polis has been used in a variety of different contexts as part of broader political processes facilitating broader political participation and opinion-forming, and gathering feedback and creative input.

Use of Polis in Taiwan

One prominent use case of Polis is its adoption as part of the vTaiwan participatory governance project. Established by the g0v civic tech community in the wake of the 2014 mass protests by the Sunflower movement, the vTaiwan project enables consultations on proposed legislation among a broad range of stakeholders including government ministries, lawmakers, experts, interest groups, civil society as well as the broader public. Although the resulting recommendations are non-binding, they exert pressure on the government to take action and recommendations have been adopted into legislation.345

vTaiwan uses Polis for large-scale online deliberations as part of a structured participation process. These deliberations take place after identifying and involving stakeholders and experts and providing through information about the topic at hand to the public. Citizens are then given the opportunity to vote on statements or provide alternative proposals which allows for the refinement of ideas and ideally leads to a consensus at the end. The results of these online deliberations are then curated, discussed in publicly broadcast face-to-face meetings which ultimately produce concrete policy recommendations. vTaiwan has in numerous cases given impulses resulting in government action and provided significant input e.g. on legislation regulating Uber or technological experiments by Fintech startups.35

See also

  1. https://compdemocracy.org/Polis/ 

  2. https://compdemocracy.org/comment-routing/ 

  3. https://info.vtaiwan.tw/ 

  4. https://www.theguardian.com/world/2020/sep/27/taiwan-civic-hackers-polis-consensus-social-media-platform 

  5. https://www.technologyreview.com/2018/08/21/240284/the-simple-but-ingenious-system-taiwan-uses-to-crowdsource-its-laws/ 

02 July, 2024 02:14PM by sunweaver

hackergotchi for Colin Watson

Colin Watson

Free software activity in June 2024

My Debian contributions this month were all sponsored by Freexian.

  • I switched man-db and putty to Rules-Requires-Root: no, thanks to a suggestion from Niels Thykier.
  • I moved some files in pcmciautils as part of the /usr move.
  • I upgraded libfido2 to 1.15.0.
  • I made an upstream release of multipart 0.2.5.
  • I reviewed some security-update patches to putty.
  • I packaged yubihsm-connector, yubihsm-shell, and python-yubihsm.
  • openssh:
    • I did a bit more planning for the GSS-API package split, though decided not to land it quite yet to avoid blocking other changes on NEW queue review.
    • I removed the user_readenv option from PAM configuration (#1018260), and prepared a release note.
  • Python team:
    • I packaged zope.deferredimport, needed for a new upstream version of python-persistent.
    • I fixed some incompatibilities with pytest 8: ipykernel and ipywidgets.
    • I fixed a couple of RC or soon-to-be-RC bugs in khard (#1065887 and #1069838), since I use it for my address book and wanted to get it back into testing.
    • I fixed an RC bug in python-repoze.sphinx.autointerface (#1057599).
    • I sponsored uploads of python-channels-redis (Dale Richards) and twisted (Florent ‘Skia’ Jacquet).
    • I upgraded babelfish, django-favicon-plus-reloaded, dnsdiag, flake8-builtins, flufl.lock, ipywidgets, jsonpickle, langtable, nbconvert, requests, responses, partd, pytest-mock, python-aiohttp (fixing CVE-2024-23829, CVE-2024-23334, CVE-2024-30251, and CVE-2024-27306), python-amply, python-argcomplete, python-btrees, python-cups, python-django-health-check, python-fluent-logger, python-persistent, python-plumbum, python-rpaths, python-rt, python-sniffio, python-tenacity, python-tokenize-rt, python-typing-extensions, pyupgrade, sphinx-copybutton, sphinxcontrib-autoprogram, uncertainties, zodbpickle, zope.configuration, zope.proxy, and zope.security to new upstream versions.

You can support my work directly via Liberapay.

02 July, 2024 12:02PM by Colin Watson

hackergotchi for Junichi Uekawa

Junichi Uekawa

July.

July. My recent coding was around my AC controls at home. That's about it.

02 July, 2024 07:17AM by Junichi Uekawa

hackergotchi for Ben Hutchings

Ben Hutchings

FOSS activity in June 2024

02 July, 2024 01:46AM by Ben Hutchings

FOSS activity in May 2024

02 July, 2024 12:08AM by Ben Hutchings

July 01, 2024

FOSS activity in April 2024

01 July, 2024 11:18PM by Ben Hutchings

Paul Wise

FLOSS Activities June 2024

Focus

This month I didn't have any particular focus. I just worked on issues in my info bubble.

Changes

Issues

Communication

  • Respond to queries from Debian users and contributors on IRC

Sponsors

All work was done on a volunteer basis.

01 July, 2024 10:16PM

Sahil Dhiman

Personal ASNs From India

Internet and it’s working are interesting and complex. We need an IP address to connect to the Internet. A group of IP addresses with common routing policy is known as an Autonomous System (AS). Each AS has a globally unique Autonomous System Number (ASN) and is maintained by a single entity or individual(s). Your ISP would have an ASN. IP addresses/prefixes are advertised (announced) by an AS through Border Gateway Protocol (BGP) to its peers (ASes which it connects to) to steer traffic in its direction or back.

Take for example Google DNS service at 8.8.8.8 owned and operated by AS15169 Google LLC. AS15169 through BGP announcements, lets all its peers know that traffic for whole of 8.8.8.0/24 (including 8.8.8.8) prefix should be sent to them. See the following screenshot response of mtr -zt 8.8.8.8 from my system. From my Internet Service Provider (ISP), AS133982 Excitel Broadband, traffic travels to AS15169 to reach 8.8.8.8 (dns.google) and returns via the same path. This Inter-AS traffic makes the Internet tick.

mtr from Excitel to Google

ASes comes in different sizes and purposes. Like AS749 DoD Network Information Center which holds more than 200 million+ IPv4 addresses for historical reasons or AS23860 Alliance Broadband Services which has 68 thousand+ IPv4 address for purpose of providing consumer Internet.

Similarly, some individuals also run their personal ASN including a bunch of Indians. Most of these Indian ASNs are IPv6 (primary or only) networks run for hobby and educational purposes. I was interested in this data, so complied a list of active ones (visible in the global routing table) from BGP.Tools:

Let me know if I’m missing someone.

01 July, 2024 07:45PM

Niels Thykier

Debian packaging with style black

When I started working on the language server for debputy, one of several reasons was about automatic applying a formatting style. Such that you would not have to remember to manually reformat the file.

One of the problems with supporting automatic formatting is that no one agrees on the "one true style". To make this concrete, Johannes Schauer Marin Rodrigues did the numbers of which wrap-and-sort option that are most common in https://bugs.debian.org/895570#46. Unsurprising, we end up with 14-15 different styles with various degrees of popularity. To make matters worse, wrap-and-sort does not provide a way to declare "this package uses options -sat".

So that begged the question, how would debputy know which style it should use when it was going to reformat file. After a couple of false-starts, Christian Hofstaedtler mentioned that we could just have a field in debian/control for supporting a "per-package" setting in responds to my concern about adding a new "per-package" config file.

At first, I was not happy with it, because how would you specify all of these options in a field (in a decent manner)? But then I realized that one I do not want all these styles and that I could start simpler. The Python code formatter black is quite successful despite not having a lot of personalized style options. In fact, black makes a statement out of not allowing a lot of different styles.

Combing that, the result was X-Style: black (to be added to the Source stanza of debian/control), which every possible reference to the black tool for how styling would work. Namely, you outsource the style management to the tool (debputy) and then start using your focus on something else than discussing styles.

As with black, this packaging formatting style is going to be opinionated and it will evolve over time. At the starting point, it is similar to wrap-and-sort -sat for the deb822 files (debputy does not reformat other files at the moment). But as mentioned, it will likely evolve and possible diverge from wrap-and-sort over time.

The choice of the starting point was based on the numbers posted by Johannes #895570. It was not my personal favorite but it seemed to have a majority and is also close to the one suggested by salsa pipeline maintainers. The delta being -kb which I had originally but removed in 0.1.34 at request of Otto Kekäläinen after reviewing the numbers from Johannes one more time.

To facilitate this new change, I uploaded debputy/0.1.30 (a while back) to Debian unstable with the following changes:

  • Support for the X-Style: black header.
  • When a style is defined, the debputy lsp server command will now automatically reformat deb822 files on save (if the editor supports it) or on explicit "reformat file" request from the editor (usually indirectly from the user).
  • New subcommand debputy reformat command that will reformat the files, when a style is defined.
  • A new pre-commit hook repo to run debputy lint and debputy reformat. These hooks are available from https://salsa.debian.org/debian/debputy-pre-commit-hooks version v0.1 and can be used with the pre-commit tool (from the package of same name).

The obvious omission is a salsa-pipeline feature for this. Otto has put that on to his personal todo list and I am looking forward to that.

Beyond black

Another thing I dislike about our existing style tooling is that if you run wrap-and-sort without any arguments, you have a higher probability of "trashing" the style of the current package than getting the desired result. Part of this is because wrap-and-sort's defaults are out of sync with the usage (which is basically what https://bugs.debian.org/895570 is about).

But I see another problem. The wrap-and-sort tool explicitly defined options to tweak the style but provided maintainers no way to record their preference in any machine readable way. The net result is that we have tons of diverging styles and that you (as a user of wrap-and-sort) have to manually tell wrap-and-sort which style you want every time you run the tool.

In my opinion that is not playing to the strengths of neither human nor machine. Rather, it is playing to the weaknesses of the human if anything at all.

But the salsa-CI pipeline people also ran into this issue and decided to work around this deficiency. To use wrap-and-sort in the salsa-CI pipeline, you have to set a variable to activate the job and another variable with the actual options you want.

The salsa-CI pipeline is quite machine readable and wrap-and-sort is widely used. I had debputy reformat also check for the salsa-CI variables as a fallback. This fallback also works for the editor mode (debputy lsp server), so you might not even have to run debputy reformat. :)

This was a deliberate trade-off. While I do not want all us to have all these options, I also want Debian packaging to be less painful and have fewer paper cuts. Having debputy go extra lengths to meet wrap-and-sort users where they are came out as the better solution for me.

A nice side-effect of this trade-off is that debputy reformat now a good tool for drive-by contributors. You can safely run debputy reformat on any package and either it will apply the styling or it will back out and inform you that no obvious style was detected. In the latter case, you would have to fallback to manually deducing the style and applying it.

Differences to wrap-and-sort

The debputy reformat has some limitations or known differences to wrap-and-sort. Notably, debputy reformat (nor debputy lsp server) will not invoke wrap-and-sort. Instead, debputy has its own reformatting engine that provides similar features.

One reason for not running wrap-and-sort is that I want debputy reformat to match the style that debputy lsp server will give you. That way, you get consistent style across all debputy commands.

Another reason is that it is important to me that reformatting is safe and does not change semantics. This leads to two regrettable known differences to the wrap-and-sort behavior due to safety in addition to one scope limitation in debputy:

  1. debputy will ignore requests to sort the stanzas when the "keep first" option is disabled (-b --no-keep-first). This combination is unsafe reformatting. I feel it was a mistake for wrap-and-sort to ever allow this but at least it is no longer the default (-b is now -bk by default). This will be less of a problem in debhelper-compat 15, since the concept of "main package" will disappear and all multi-binary source packages will be required to use debian/package.install rather than debian/install.
  2. debputy will not reorder the contents of debhelper packaging files such as debian/install. This is also an (theoretical) unsafe thing to do. While the average package will not experience issues with this, there are rare corner cases where the re-ordering can affect the end result. I happen to know this, because I ran into issues when trying to optimize dh_install in a way that assumed the order did not matter. Stuff broke and there is now special-case code in dh_install to back out of that optimization when that happens.
  3. debputy has a limited list of wrap-and-sort options it understands. Some options may cause debputy to back out and disable reformatting entirely with a remark that it cannot apply that style. If you run into a case of this, feel free to file a feature request to support it. I will not promise to support everything, but if it is safe and trivially doable with the engine already, then I probably will.

As stated, where debputy cannot implement the wrap-and-sort styles fully, then it will currently implement a subset that is safe if that can be identified or back out entirely of the formatting when it cannot. In all cases, debputy will not break the formatting if it is correct. It may just fail at correcting one aspect of the wrap-and-sort style if you happen to get it wrong.

It is also important to remember that the prerequisite for debputy applying any wrap-and-sort style is that you have set the salsa-CI pipeline variables to trigger wrap-and-sort with the salsa-CI pipeline. So there is still a CI check before the merge that will run the wrap-and-sort in its full glory that provides the final safety net for you.

Just give me a style

In conclusion, if you, like me, are more interested in getting a consistent style rather than discussing what that style should be, now you can get that with X-Style: black. You can also have your custom wrap-and-sort style be picked up automatically for drive-by contributors.

$ apt satisfy 'dh-debputy (>= 0.1.30), python3-lsprotocol'

# Add ``X-Style: black`` to ``debian/control`` for "just give me a style"
#
# OR, if there is a specific ``wrap-and-sort`` style for you then set
# SALSA_CI_DISABLE_WRAP_AND_SORT=no plus set relevant options in
# SALSA_CI_WRAP_AND_SORT_ARGS in debian/salsa-ci.yml (or .gitlab-ci.yml)

$ debputy reformat

It is sadly not yet in the salsa-ci pipeline. Otto is looking into that and hopefully we will have it soon. :)

And if you find yourself often doing archive-wide contributions and is tired of having to reverse engineer package formatting styles, consider using debputy reformat or debputy lsp server. If you use debputy in this way, please consider providing feedback on what would help you.

01 July, 2024 10:15AM by Niels Thykier

hackergotchi for Guido Günther

Guido Günther

Free Software Activities June 2024

A short status update of what happened on my side last month. Was able to test our Cellbroadcast bits, feedbackd became more flexible regarding LEDs, Phosh 0.40 is out, and some more.

Phosh

  • Phosh 0.40.0
  • Fix testing custom quick setting plugins (MR)
  • Add icons for dark style toggles (thanks to Sam Hewitt for the help) (MR, MR)
  • 0.39.x backports (MR)
  • Set default sound-theme (MR)
  • Launch apps in transient scope (MR)
  • Allow to suspend from lock screen (MR)
  • Fix media player button style (MR)
  • Don't use plain .so for bindings lib (MR)

Phoc

  • Fling gesture to open/close phosh's panels (MR)
  • Fix crash on output removal (MR)
  • Don't draw decorations when maximized (MR)
  • Allow to stack layer surfaces (MR was a bit older, polished up to land it) (MR)

gmobile

  • Add hwdb support for Juno tablets (based on information by Giovanni Caligaris) (MR)
  • Support add generic BT AVRCP profile (with the help of Phil Hands) (MR)
  • Released 0.2.1

phosh-mobile-settings

  • Use shared gmobile (MR)

phosh-wallpapers

  • Add sound theme (MR)

emacs

  • Add a major mode for systemd-hwdb files (MR)

Debian

  • Backport phog fix to work with phosh-osk-stub (MR)
  • Release git snapshot of phosh-wallpaper for NEW processing (MR)
  • Backport phosh fixes for 0.39.0 (MR)
  • Phoc: Install examples, they're useful for debugging (MR)
  • Make libpam-ccreds work with gcrypt 1.11:
  • Upload phosh release 0.40.0~rc1 and 0.40.0
  • phog: Add example for autologin (MR)
  • Update firefox-esr-mobile-config (thanks to Peter Mack for the input) (MR)
  • Tweak meta-phosh:
    • Add feedbackd-device-themes (MR)
    • Add jxl pixbuf loader (MR)
    • Make gstreamer-packagekit a recommends (MR)

ModemManager

Feedbackd

  • Only apply Qualcom bits to lpg driver (MR)
  • Support arbitrary RGB values for multicolor LEDs (MR)
  • Allow to use camera flash LEDs as status LEDs (MR)
  • End too noisy feedback when switching profile levels (MR)
    • calls bugfix for this (MR)
    • Trigger this via vol- in Phosh when a call comes in (MR)
  • Packaging fixes (MR) , MR)
  • device-themes: Lower brightness of feedback events as the flash is too bright on OnePlus 6T (MR)
  • cli: Inform user when nothing was triggered (MR)
  • Released 0.4.0

Livi

  • Released 0.2.0
  • Robustify stream resume (MR)

Calls

  • build: Add summary (MR)
  • Handle missing sim better (MR)
  • Backports for 46 (MR)
  • Fix SIP crash (MR)

Chatty

  • Allow to display a Matrix clients access token (MR)
  • libcmatrix: Add support handling push notification servers (MR)
  • Allow to add push notification servers (draft)) (MR)
  • Package docs (MR)

meta-phosh

  • Add check for release consistency (MR)
    • mobile-settings: Use it in mobile settings (MR)

Libhandy

  • Fix use-after-free in stackable-box (MR)

If you want to support my work see donations.

01 July, 2024 08:01AM