Reporting functionality: Discussion and clarity

Can the design and product team get responses to these

I have realized in reading @eriol’s message that I cannot confidently say who comprises this team.

My uncertainty here makes me think we need to do a better job documenting who is responsible for different aspects of Hypha’s development. Some of the info exists on the Hypha main page and the main documentation page, but we can make more explicit the division of labor (in the sense of saying “here’s who to contact if you have questions about:”).

It might also be a good idea to list the means by which we can contact each of those folks (perhaps a link to WeChat and their handle on the platform).

Questions for folks:

  • Do you agree that we should make this more explicit to the community? (For ease of generating consensus, please like the post if you agree)
  • Where should it go (main page only, main page + main docs page)?

Once we have answers to those two questions, I can create GitHub tickets in the relevant repository/repositories to begin the process.

1 Like
  1. What kinds of reports do you currently ask for from applicants? (e.g. written long form, written short, automated surveys via SMS or email etc.)

Every 6 months we do an informal check in via a Zoom, phone call, or email. In this case the report end product is my notes on the call or the text of the email update.

When the project is completed they fill out this form: ARDC Final Grant Report

Every project is different, and we ask for different things at the 1-year mark. For example, a large satellite project may require a written report on each deliverable, whereas a scholarship grant may (a) be done already and (b) just need a list of recipients, where they went to school, etc.

  1. What kinds of questions do the applicants answer in these reports and to what extent do you value different kinds of questions/responses? (e.g. quantitative, qualitative and you value each equally)

You can view the questions here: ARDC Final Grant Report

Question type value depends on the project, so for the purposes of building reporting in Hypha, both are equally valuable.

For example, for scholarships, the number and amount of scholarships distributed by our partners is the most important measure of success. For a project where a club is putting up a repeater antenna, describing that they successfully installed the antenna is the most important part.

  1. What kinds of methods or tools do you either ask applicants to use to send reports or do applicants request to use to send you reports? (e.g.attachment files on emails, SMS on Signal etc.)

Attachments on email is fine, but we prefer to use a form (which right now is in Airtable). The ability to upload attachments is helpful for photos and people who have long publications they want to upload.

  1. What kind of follow-up do you do as a funder to reports from applicants? (e.g. we offer support, sign post for support from other places etc.)

We use reports for:

  • Publicizing information in our annual report, in blog posts, and at public meetings where we share about the work we’ve been doing. Please note that we ask specifically for information that can be shared publicly. This may be a subset of the report, or the whole thing, or none of it.

  • We use internal report info about lessons learned and challenges to talk with our board and reviewers about which projects need additional support, or which types of projects we should think about funding in the future

  • We would like to use reports to advise new applicants about what makes a project successful and common pitfalls

2 Likes

@allan when you have a chance do you think you could answer these questions from a DDP pov?

  1. What kinds of reports do you currently ask for from applicants? (e.g. written long form, written short, automated surveys via SMS or email etc.)

  2. What kinds of questions do the applicants answer in these reports and to what extent do you value different kinds of questions/responses? (e.g. quantitative, qualitative and you value each equally)

  3. What kinds of methods or tools do you either ask applicants to use to send reports or do applicants request to use to send you reports? (e.g.attachment files on emails, SMS on Signal etc.)

  4. What kind of follow-up do you do as a funder to reports from applicants? (e.g. we offer support, sign post for support from other places etc.)

Adding in the document from ARDC re. impacts that they shared last Adopters meeting!

@blah You mentioned in maintainers last week you had a few reports centric issues you wanted to link for me ahead of the feasibility review meeting [Link pending] for reporting

Hi @Adopters and @Implementers, here’s a video (it lasts about 20 minutes) covering the reporting feature idea, the concepts, how it could work.

The video covers:

  • What do we mean by reporting?
    • Reporting on project impact and progress
    • Using the terms “reporting” and “project update”
    • Fundees Reporting To The Funders
    • Funders Reporting To Their Funders
    • Funders Reporting Internally
    • Hypha System Performance
    • What we’re focusing on first
  • Project update concepts
    • Areas of focus
    • Hypotheses
    • Meeting the funders goals/missions
  • Mapping the deliverables to area of focus mapping
  • How the reporting process could work:
    • What is the purpose of the update?
    • Who is making the update?
    • How is the update being made?
    • Who can read/see the update?
    • What is the update frequency?
    • When does the update process start?
  • How the evaluating/visualising reporting information aspect could work
    • Visualising project impact and progress 1) individually and 2) on aggregate

Thoughts, opinions, constructive criticism and feedback on what’s covered. I’m experimenting with the format, and the way I’m communicating this work, so if let me know if it does/doesn’t work for you.

3 Likes

Works for me Bernard

1 Like

Thanks @bernard ! This is broadly in line with what DFF is doing so if we can implement something similar in Hypha that would be great. I like the video format as we can go to it in our own time, rather than having to set aside time for a call.

2 Likes

Thanks for the feedback @vinkthom. Glad that it is in line with how DFF works.

Apologies for not replying to this, getting swamped with the product work.

I would like to discuss this on the next docs WG call as i think it’s a connected topic to this work but not about reporting specifically. - I’ll add to the agenda for Documentation Working Group Meeting 30th Nov

FYI @bernard Allan got back to me in a PM and said:

I have collated some answers and I’ll send them across to you by the end of this week. Its been super busy the last weeks on my end and very little to do with hypha.

So we’ll have the DDP perspective as soon as Allan gets some breathing space from day to day work

Updated feature scoping document here: Feasibility Review Template - M&E and Reporting - Adopters Nov & Dec 2021 - Google Sheets

Video recording explaining these different ideas here: WIP Pedning

@bernard Allan has shared his thoughts re. reports in a password protected doc - I’ll send this to you in a DM

@eriol Before I can estimate the dev hours I need a discussion of what the features entails more in detail. E.G 1.2.1 can be as simple as a URL field to paste in links to other documents all the way up do a complex document handling system that can index and create links to sections in doc and pdf files.

Same with 1.2, bringing in all the different inputs can become immensely complex.

[Triggered by @dluong’s mention of adding category questions to reporting in the reporting Miro board]

Can someone (I’m thinking @blah or @dluong or @frjo) explain the original purpose of “category questions” in Hypha? What need were they created to address?

I understand that the are questions which the applicant are required to answer, but I don’t have a full understanding of how they are used in practice.

When they are used, what is done with the answers? What can the answers be used for?

1 Like

There was a need to have question of the type “Country” and “Focus” etc. that all had multiple options and that should be the same on many/all applications.

Recreating e.g. a “What country are you from” question on each application with the option to pick any of the worlds 195 countries would be cumbersome.

With “Category question” they can be setup once and then reused on any number of application.

Since they are the same on all application they can be used to collect statistics etc.

@bernard I added a category question to this form if you want to reverse engineer it:
Form: Sign in ・ Cloudflare Access
Category Question: Sign in ・ Cloudflare Access

A category question can look like this:

Hypha chews up the data and Fredrik spits it out as a spreadsheet. Some poor sap made this:

Some talentless hack made this category question:

1 Like

Hypha @Adopters, I’ve put together some early design ideas for how funders can define their organisations areas of focus in Hypha.

The purpose of this (optional) step is to allow funders to map projects, and their deliverables/milestones, to the funders areas of focus. This would give them more granular understanding on what areas of focus they’re funding more, the kinds of projects they’re funding to make impact in those areas, and other insights.

The walk through video lasts about 18 mins.

If you’d like to leave comments or feedback, please do so here, or in the Miro board itself.

Meta terms are unsuitable for areas of focus. We should add a new separate setting for them. This makes it unambiguous where the list reside.

Added as meta terms Hypha does not know what was areas of focus and other terms.

Option 2 is therefor my preference.

These will then be applied to each deliverable in projects, is that correct? If one deliverable covers two or more areas of focus, do we need to have a setting for percent per area as well? Area 1 30%, area 2 50% and area 3 20%.

Then Hypha can accurately report how much money have been spent on each area. And that is the point of this feature as I understand it.

This will be quite a lot of work for staff to set up for each project but you will get nice numbers on the other end.