AA SIG - Audio HAL - Meeting Minutes
PoC Milestones and Work Breakdown
AASIG (Audio-HAL) Workshop at 20th All Member Meeting (agenda planning)
List of prioritized topics for the Audio HAL
Multiple-zone audio management - System Level Audio
Next Meeting - TBD 2021 (summer vacations pause) - 11:30am CEST
Click to Join Webex meeting
or join by entering the meeting details manually:
Meeting number (access code): 297 637 101
Meeting password: XCtdZmvs248
Or join the meeting by phone with
Access code: 297 637 101
Meeting password: 92839687
Agenda
Discuss design of audio→external system, WebRTC based demo and "desktop audio development environment"
Thursday 15 July - 11:30am CEST
Participants
@Gunnar Andersson @Former user (Deleted) @Suhasini Raghuram (Unlicensed)
Apologies
@Piotr.Krawczyk (Deactivated) @Philippe Robin @Stephen Lawrence @Former user (Deleted)
Minutes
RPC interface. What is in there? Is it volume, settings, ...?
Gunnar: Shared work to define this. Let's set up a Wiki page to list all the required methods and get/settable properties.
Gunnar: I have also mentioned the possibility to use the CVII:VSC IDL to describe the interface, but this is a secondary goal and not where we need to start.
Wassim shows a lot of interest in this, the development and use of a standard IDL, and has experience working on topics of IDL, format-conversions, documentation, etc.
How to transfer PCM data between "Host" and VM? Is it the socket approach as discussed before?
Thursday 8 July - 11:30am CEST
Participants
@Piotr.Krawczyk (Deactivated) @Gunnar Andersson @Stephen Lawrence @Philippe Robin
Apologies
@Former user (Deleted) ( + 1 more week )
Minutes
Discussed content of AMM demo See PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control)
Piotr indicates the Android side can be done in time. On host side we expect to have something, scope to be determined.
Piotr to send description of project/plan to aasig mailing list and ask Wassim for input on the host-side of the plan.
Thursday 1 July - 11:30am CEST
Participants
@Suhasini Raghuram (Unlicensed)@Piotr.Krawczyk (Deactivated) @Gunnar Andersson
Apologies
@Former user (Deleted) (+ 2 more weeks)
Minutes
The full diagram of discussed plans has been posted by Piotr.
It includes Audio-offloading and desktop development environment ideas, as discussed.
Thursday 24 June - 11:30am CEST
Minutes
Very short meeting due to apologies and absences
First draft diagram by Piotr coming via email. To be shown and discussed.
Thursday 17 June - 11:30am CEST
Participants
@Former user (Deleted) @Suhasini Raghuram (Unlicensed) @Philippe Robin @Piotr.Krawczyk (Deactivated) @Former user (Deleted)
Minutes
Discussion about the work
The main summary is that we need more people to implement
Discussion about PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control)
Thanks to Wassim for making this page somewhat up to date
The talk is more oriented towards the big picture
The big picture: Shifting from Android as device centric based operating system (like with the smartphone) to car based operated system
Then other topics would be in that direction
for example extracting the audio, effects, offloading, control, Bluetooth source, etc.
we shifted from what's needed for the features that we want to implement to what is needed to implement that direction
Where would be have development and programmers?
We worry less and less about what Android is doing when we go to the direction of the border of the OS
Concrete steps for next week. 2 major tasks:
Making the documentation and diagrams of what we intend to do (check the WBS)
Track related features in new android versions to check that there are no surprises or something that would block us
Please check first with your NDA not to break it
Goal of the development
Should be more concrete than the big picture such as
configuration of audio zones
environment for building the framework inside the system
environment for building the framework outside the system
Piotr: I would start with high level design for these tasks first
like controlling audio zones, volumes, context
design of trout based development (from an architecture point of view of course this is not important)
but does trout support audio?
it's a nice basis because the framework will not change, so the development can be taken to other HW easily. for example in an emulator, the framework was changed
cross VM is an advantage (it's used in Trout)
closer to what we want to develop in future
closer to real use case
This is where the trend is going
get a portability of Android with trout (but not other OS)
we need to check what is the real platform that we would work with in the future
we need to check the good reasons to select it
Wassim: I would focus more on policy and logic management specially for our GENIVI group
Focusing also on versions of Android in the background and what would they offer
Thursday 10 June - 11:30am CEST
Participants
@Former user (Deleted) @Suhasini Raghuram (Unlicensed) @Gunnar Andersson @Philippe Robin
apologies: @Former user (Deleted)
Minutes
Discussion on the workbreakdown structure, PoC Milestones and Work Breakdown (AASIG, AHAL, audio-control) updated online
Thursday 27 May 2021 - 11:30am CEST
Participants
@Former user (Deleted) @Gunnar Andersson @Philippe Robin
apologies: @Former user (Deleted)
Minutes
Discussion on the project organization and workplan and on how to foster engineers' engagement in day2day work, outcome of the discussion will be reported in the next call
Thursday 20 May 2021 - 11:30am CEST (AudioHAL CW20)
Participants
@Gunnar Andersson @Suhasini Raghuram (Unlicensed) @Former user (Deleted) @Stephen Lawrence @Philippe Robin
Minutes
Small discussion about the AMM
Interest in virtualization, trout, virtio
Discussion about virtualization, virtio, networking virtualization, TSN area in the hypervisor group
Discussion about the implementation
Have we moved away from topics that are interesting for Analog Devices? A2B connected technologies
Maybe we can get back to these topics when we are designing hardware POCs for the virtualization
There some interests of developing Linux drivers for A2B not only Android
Also standardized way of connecting A2B
Idea of diagnostic protocol as a carrier for end user functionalities
no call on Thursday 13 May (due to a bank holiday)
May 4-7 Virtual AMM - AUDIO HAL sessions
5 May - technical update: recorded session (please go to AUDIO HAL section), Android Automotive Audio slides
6 May - technical workshop #1: Recorded Session (please go to AUDIO section),
7 May - technical workshop #2: Recorded Session (please go to AUDIO section)
Thursday 29 April 2021 - 11:30am CEST
Participants
@Philippe Robin @Gunnar Andersson @Piotr.Krawczyk (Deactivated) @Suhasini Raghuram (Unlicensed) @Former user (Deleted)
apologies @Former user (Deleted) @Stephen Lawrence
Minutes
preparation of AMM last session - discussion on the topics for next 6 months
virtualization => how can we minimize changes to Android (virtio, trout)
simulation => how can we simulate HW acceleration in a seemless environment together an Emulated Android
are we chasing after Android 12 or are we ahead of Android for the external source management in an automotive environment ?
Thursday 22 April 2021 - 11:30am CEST (AUDIO_HAL CW2116)
Participants
@Philippe Robin @Gunnar Andersson @Piotr.Krawczyk (Deactivated) @Suhasini Raghuram (Unlicensed) @Former user (Deleted) @Former user (Deleted) @Stephen Lawrence
Minutes
Synchronizing from last weeks' status
Wassim: We presented some suggetsions and discussed how they can help, what are your impressions?
Piotr: Audio Weaver was specially a quite interesting topic and it seems widely used.
Wassim: you can use Audio Weaver for free with https://elinux.org/R-Car/Boards/M3SK development kit
Wassim: you can use it on PC but it's based on Matlab
Wassim: the goal is the smooth transition between the demo/emulation, and the product. Concept is requirement as product model.
In the future a researcher would just for example provide an effect model and this would be a requirement to the suppliers
Specially that we are using Android. We can have a ready Android system, and a ready effect model. What is left is to connect them.
Wassim: this is the picture posted here Car Audio System Emulator. I'll take some time to create ppt Genivi slides
more discussion about the demo but with WebRTC and Trout
Piotr: It's a nice this audio solution (refering to the Car Audio System Emulator). To have the effects controlled with a model outside of Android.
Piotr: In the end, we could have the WebRTC interface showing Audio coming out of Android and Effect controlled by the Audio Effect HAL and allows the user to apply additional effects on top of the effects.
Wassim: yes and we could also have them decoupled: PC simulation where the decoupling happen but browser is GUI.
Piotr: one question is about the LibPulse Audio
Wassim: libpulse audio is simply a tool. What is more important is to show:
if we have in the car many zones/speakers/etc.
the user would see it on his browser the streams the channels etc.
the user would choose what to listen to for example
the architecture of the Car Audio System Emulator is independent of the PC but can be patched and used to control
Pitor: further question: should the ECNR control interface be exposed? and to what level are they exposed?
Wassim: as long as Android allows to do it, should be enough.
Wassim: as long as something can be controlled from hardware, it should be controllable by software
Wassim: imagine that every app has its own ECNR and multiple applications would need to run at the same time → this is why it's more interesting to have a system ECNR
Pitor: common solutions don't utilize DSP firmware by default
Piotr: if we talk about Trout , it's not an emulator, it's just an Android instance and most of the HAL will be virtualized. We are not connected to the pulse audio here for example. to keep with the same picture, we would do something different
Wassim: completely okay, we don't have to use the pulseaudio server
Piotr: we can think about not using ALSA, because of the virtualization.
Piotr: this virtualization would be separate from browser and ALSA.
Piotr: android emulator - virtual android
Pitor link for trout and virtualization https://source.android.com/devices/automotive/virtualization
Wassim: we didn't solve the control path yet
Wassim: between the android and browser how would you see that? Json?
Piotr: audio control already implemented in trout, let's check a solution in that direction. Since Trout already has solutions for several audio control
Wassim: we need to look how easy it is to deploy the grpc on the browser
Wassim: 3 demonstrations so far: raw audio with float sample packets, Opus Codec, WebRTC not yet fully running but it's client 2 client (looking like an overkill)
Piotr: I would say for demonstration purpose, whatever is currently working would be accepted,
Jira ticket AASIG-124
We need to edit the diagram and the names and rework it and that should be the demo concept.
Audio weaver: candidate for virtual effect platform to start a professional demo. Later we can work with python audio effect but this needs more work but is open standard.
Genivi virtual meeting 04-07 May 2021
Need to gather some topics for next meeting (Audio HAL CW 2117 - 29.04.2021)
Webex is preffered as a meeting software.
Last time some people could not join (because the meeting was hosted with Zoom)
Thursday 15 April 2021 - 11:30am CEST (AUDIO_HAL CW2115)
Participants
@Philippe Robin @Gunnar Andersson @Piotr.Krawczyk (Deactivated) @Stephen Lawrence @Suhasini Raghuram (Unlicensed) @Former user (Deleted)
Minutes
Overview of the last weeks
Piotr: status from last week regarding the demo (part of the AASIG-93 but with a twist)
Demo is working on Trout
WebRTC is now being connected
Confident to be able to show something in two weeks
Expecting more commits in https://github.com/SoundHacking/web_py_loop
Got more support about this task from Tieto → Output should be shown
Gunnar: So where are we now exactly?
Piotr: Next week we would have something to show and have a better overview of what we have and what is missing.
Gunnar: we need to check with Wassim what was his vision in order not to have an overlap and re-use different parts → Todo
Gunnar: Main idea was to create a development environment where we can try different audio solutions but the heart of this would still be Android → todo capture this in Jira
Suhasini: 3 questions about the virtual meeting 04-07 May 2021
What are the other slots used for (for example vehicle data related)?
What will the Audio HAL do in the slot exactly?
What updates are we planning on giving in the introduction
Gunnar/Philippe: the slots are being now finalized we'll update you asap. For now:
One slot is on Wednesday (05.05.2021) 5 minutes introduction
Time slot 15min about the External Audio Demonstration
Time slot workshop 1 Q&A Audio design discussion 25min
Workshop 2 (Wassim moderating) on Friday (07.05.2021) 15:25
Introduction shall be about what we have done, what we are doing, and if you like to join. Prerecorded video. At some point you will be contacted by Tom (Audio/Video/Networking) between 27 and 28 April 2021
Fosdem update: we could bring it up in the introduction on Wednesday
Stephen: DSP concept from last week. Asked internally
There is HIFI support but it's proprietary
Audio Weaver is supported in Renesas
Todo
@Former user (Deleted): To synchronize with @Piotr.Krawczyk (Deactivated) about the vision and re-usability
@Former user (Deleted) : create from the past minutes of meetings a Jira ticket and add it to the sprint
Thursday 8 April 2021 - 11:30am CET
Participants
@Stephen Lawrence @Gunnar Andersson @Former user (Deleted) @Philippe Robin @Suhasini Raghuram (Unlicensed)
Apologies
@Former user (Deleted)
Wassim has investigated webrtc with some simple implemementations
Wassim: This could be transformed into a "proper" project. Maybe hosted on github/GENIVI?
https://github.com/SoundHacking/web_py_loop – @Gunnar Andersson to review if this fits into GENIVI repos.
Documentation here: https://www.homesmartmesh.com/docs/sound/
Wassim describes two designs, one on WebRTC, and another which is more decoupled and simply uses fixed parameters for the encoding. In an environment where the bandwidth is assumed to be more controlled and less variable, then the dynamic features of WebRTC are not needed.
Both ideas are implemented and can be compared. All examples use the 00_backend (loopback) as server except for example 07 which uses 00_signaling server. Wassim to update readmes.
Interesting tool https://w.dspconcepts.com/audio-weaver
Development tool with target framework to execute the scenarios (on-board firmware)
Audio experts make a flow using GUI tool, by connecting and configuring processing blocks.
Not open-source. Likely business model is target license-fee for production use
Some DSPs, are supported.
For ST Electronics hardware, the license is included.
An inexpensive ST Microelectronics dev. board can be purchased and used directly: STM32f407 user guide.
(but there is also the option to run it on PC only)... discussion about which other DSP hardware platforms might be supported. This is not clear on the web site (need to contact the company?).
Wassim says he knows it supports production grade DSPs, but apparently these are not public information(?), so we cannot describe it further.
Stephen: I found that Cadence is listed, that's all
You can generate the config/script and send it to the target systems.
The target systems have an audioweaver DSP code.
Documentation is available.
Suhasini: Can the tool run the scenarios on the PC (a simulator/emulator environment)?
Wassim: Yes, all "blocks" can be executed on the PC with a "x86 target"
It might be interesting to support our "development environment" (described above and in minutes 2 weeks ago) integrated with this?
Wassim: (bottom line) Whether it is exactly this one or another, this type of development environment would be ideal.
Thursday 1 April 2021 - 11:30am CET
Participants
@Gunnar Andersson @Former user (Deleted) @Philippe Robin @Suhasini Raghuram (Unlicensed)
Apologies
@Former user (Deleted)
Minutes
review of the planning of presentations at the upcoming AMM
TODO Wassim and Suhasini will prepare a (5 to 10' mn) report on Audio HAL project status
TODO Wassim will contact Piotr and check with him how to deliver a demo on External Audio management at the upcoming AMM
Wassim: Android 12 is now being published
Thursday 25 March 2021 - 11:30am CET (AUDIO_HAL_CW2112)
Participants
@Piotr.Krawczyk (Deactivated) @Gunnar Andersson @Stephen Lawrence @Former user (Deleted)
Apologies
@Former user (Deleted) @Philippe Robin
Minutes
Piotr discussing that the HAL implementation might just as well target Trout since it is progressing quickly
Google's virtual platform for Android "Trout", is likely to implement VIRTIO-SND
Trout is based on CrosVM (developed as part of Chromium). CrosVM is in turn based on KVM when running on Linux.
Trout is great and will make HALs portable without/less rewrite (Piotr)
...but the complexity of hardware compatibility and porting then lands on Hypervisor and Virtual Platform implementations instead (Gunnar)
(Note Automotive Virtual Platform Specification already requires VIRTIO-SND in the working-copy, to be released in new version soon)
VIRTIO-SND proposal is merged to VIRTIO master branch but no VIRTIO v1.2 document exists yet. Seems very likely to be included since it is merged.
VIRTIO-SND kernel driver exists but implementations on virtual platforms (QEMU) still lacking. Part way done on Trout it seems...
It is being developed but we can't rely that it will be done for AMM demo.
Plan is therefore to use a simplified VSOCK transfer instead, exporting the audio. (A step in the right direction since VIRTIO-SND implementations are likely to use VSOCK)
The plan remains to start building a framework for external audio and as a first implementation set up a WebRTC server running on the "host" that can consume the data from the VSOCK connection.
The usage of WebRTC also fulfils a desire from Wassim to build a flexible simulation/processing/tinkering framework that can be executed on developer machines by way of a Web browser providing the user interface. (i.e. not only production, but development tools support).
Why WebRTC?? Summarized:
Easy demo - just "send a URL to someone, they open it in web a browser". It is also convenient way to create a user interface
Bigger context: Create an abstraction of target sound environment, which may develop into a flexible (desktop) development environment for exploring/developing audio related functions.
(Gunnar added offline): The fact that "real" Android is still part of this abstract development system makes it relevant for developing and testing real audio functions. I imagine you could even do Hardware in the loop setups...
But also: WebRTC might be a possible actual "production" protocol for audio connection to a remote server. (Example use case that needs this: off-board speech processing)
Pulseaudio? Might be an easy solution to implement some of the control channel parts for the demo.
NEXT : Need to start breaking down the design into more details, included software components
1) Find existing implementations of component parts (e.g. WebRTC – @Former user (Deleted)mentioned there are implementations available, let's point to them)
2) What parts still do not exist - need to be developed
And of course: Confirm the overall demonstration use case and plan to guide the breakdown.
Thursday 18 March 2021 - 11:30am CET (AUDIO_HAL_CW2111)
Participants
@Gunnar Andersson @Philippe Robin @Stephen Lawrence @Suhasini Raghuram (Unlicensed)
Apologies
@Former user (Deleted) @Former user (Deleted)